system

Abstract

Provide a system. [Solution] a means for inputting characteristic information from witnesses; A means for generating a portrait based on input feature information; a means for displaying the generated portrait to a witness; a means for inputting feedback from witnesses; a means for modifying the caricature based on the input feedback; a means for displaying the modified portrait to the witness again; The system includes means for storing the final portrait in a database.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The process of creating sketches for criminal investigations is not limited to dedicated police officers; trained general police officers are also tasked with this task. However, this process has several problems. The main problems are that it takes time for witnesses to request corrections in real time, and witnesses are reluctant to make corrections, which results in a low level of accuracy and slows down the investigation. As a result, there is a risk of mistaken arrests (false accusations). A new system is needed to solve these problems. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides a system that can input characteristic information from witnesses and create and modify a caricature in real time. The system includes the following elements:

[0006] 1. A means of inputting characteristic information from witnesses.

[0007] 2. A means of generating a caricature based on input feature information.

[0008] 3. A means of displaying the generated portrait to a witness.

[0009] 4. A means of inputting feedback from witnesses.

[0010] 5. A means to modify the caricature based on input feedback.

[0011] 6. A means of displaying the revised portrait to the witness again.

[0012] 7. A means of storing the final portrait in a database.

[0013] Furthermore, the input feature information and feedback information are sent to a server, which then generates and modifies the sketch based on this information. This allows for immediate reflection of eyewitness feedback and the rapid creation of highly accurate sketches. This system eliminates eyewitness hesitation and time delays, significantly improving the speed and accuracy of investigations.

[0014] A "witness" is an individual who directly witnessed or was present at the scene of an incident or crime.

[0015] "Characteristic information" is information about the external characteristics of a person seen by a witness (e.g., hair color, face shape, eye shape, etc.).

[0016] A "means" is a method, apparatus, or piece of software for accomplishing a particular purpose.

[0017] A "caricature" is a depiction of a particular person's face created based on characteristic information provided by an eyewitness.

[0018] "Generation" is the process of creating new data or images based on input information.

[0019] "Display" refers to the act of visually presenting the generated portrait to a witness or a user.

[0020] "Feedback" refers to the witness providing suggestions or opinions on the generated portrait.

[0021] "Revising" refers to changing or updating an already generated portrait based on feedback from witnesses.

[0022] A "database" is a software system for systematically storing and managing information.

[0023] A "server" is a computer system that accepts and processes requests from other computers or terminals.

[0024] An "input means" is a method or device for inputting characteristic information or feedback into a computer system.

[0025] "Transmitting means" refers to a method or device for transferring input information to another component (e.g., a server).

[0026] A "redisplaying means" is a method or device for presenting the modified portrait to the witness again.

[0027] The "storing means" refers to a method or device for permanently recording the final confirmed portrait in a database or the like. [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0032] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0033] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0034] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] [First embodiment]

[0037] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0038] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0039] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0040] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0041] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0042] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0043] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0044] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0045] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0046] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0047] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0048] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[0050] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[0051] System Configuration

[0052] The system consists of three main components:

[0053] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[0054] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0055] 3. Terminal: An input device used by witnesses and police officers.

[0056] Program processing overview

[0057] 1. Initial Setup

[0058] The terminal starts up and the officer logs in.

[0059] The server verifies the login information and authentication is complete.

[0060] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0061] A message will appear on the device indicating that the initial setup is complete.

[0062] 2. Hearing from witnesses

[0063] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[0064] The terminal transmits input data to the server in real time.

[0065] 3. Caricature Generation

[0066] The server generates an initial portrait based on the received feature information.

[0067] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[0068] 4. Adjustments and feedback

[0069] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0070] The user (police officer) inputs the desired corrections into the device's UI.

[0071] The terminal transmits the correction data to the server.

[0072] The server updates the portrait based on the correction data, and transmits the updated portrait data to the terminal again.

[0073] The updated portrait will be displayed on the device, and the process will be repeated until the witness is satisfied.

[0074] 5. Final confirmation and saving

[0075] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[0076] The user (police officer) selects "Save."

[0077] The server stores the final portrait data in a database.

[0078] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[0079] Specific examples

[0080] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0081] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0082] 2. The server generates the first portrait based on the feature information and sends it to the device.

[0083] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0084] 4. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[0085] 5. The server updates the portrait based on the correction data and sends it back to the device.

[0086] 6. Repeat this process until you have created a portrait that satisfies the witness.

[0087] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0088] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0089] The processing flow will be explained below.

[0090] Step 1:

[0091] The terminal starts up and the officer enters their login information.

[0092] Step 2:

[0093] The server verifies the login information and authentication is complete.

[0094] Step 3:

[0095] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0096] Step 4:

[0097] A message will appear on the device indicating that the initial setup is complete.

[0098] Step 5:

[0099] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0100] Step 6:

[0101] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[0102] Step 7:

[0103] The terminal transmits the input characteristic information to the server in real time.

[0104] Step 8:

[0105] The server generates an initial portrait based on the received feature information.

[0106] Step 9:

[0107] The server transmits the generated portrait data to the terminal.

[0108] Step 10:

[0109] The first portrait will be displayed on the device.

[0110] Step 11:

[0111] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0112] Step 12:

[0113] The user (police officer) inputs the desired corrections into the device's UI.

[0114] Step 13:

[0115] The terminal transmits the correction data to the server.

[0116] Step 14:

[0117] The server updates the portrait based on the correction data.

[0118] Step 15:

[0119] The server transmits the updated portrait data to the terminal again.

[0120] Step 16:

[0121] The revised portrait will be displayed on the device.

[0122] Step 17:

[0123] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[0124] Step 18:

[0125] The users (witnesses and police officers) agree on the final sketch.

[0126] Step 19:

[0127] The user (police officer) selects "Save."

[0128] Step 20:

[0129] The server stores the final portrait data in a database.

[0130] Step 21:

[0131] The server notifies the terminal that the save is complete.

[0132] Step 22:

[0133] A message will appear on your device indicating that the save is complete.

[0134] Example 1

[0135] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0136] In criminal investigations, the task of creating a portrait based on eyewitness testimony is time-consuming and labor-intensive. Furthermore, eyewitnesses' memories are often vague, requiring repeated revisions, making it difficult to quickly generate a highly accurate portrait. To address these challenges, a technology is needed that can generate and revise portraits in real time based on information obtained from eyewitnesses, ultimately resulting in a highly accurate portrait.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0138] In this invention, the server includes a means for a user to input feature information from a witness, a means for generating a portrait using a generative AI model based on the input feature information, and a means for displaying the corrected portrait to the user again. This allows for the generation and correction of portraits based on witness testimony in real time, thereby improving the efficiency and accuracy of investigations.

[0139] The "user" is the person who operates the system, inputs information from witnesses, and reviews the generated and modified portraits.

[0140] A "witness" is someone who saw a suspect at a crime scene or elsewhere, and whose testimony is used to generate or modify the sketch.

[0141] "Characteristic information" refers to specific physical characteristics of a suspect as testified by witnesses (e.g., age, gender, hair color, face shape, eye shape, etc.).

[0142] The "server" is the central computing device of the system, a combination of hardware and software that receives feature information, generates and modifies portraits using generative AI models, and stores data.

[0143] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate and modify caricatures in real time using feature information as input.

[0144] A "caricature" is an image of a suspect's face generated from witness testimony and can be digitally displayed and modified.

[0145] "Feedback" is information including desired corrections and suggestions made by the witness to the generated portrait.

[0146] A "database" is a digital storage system that organizes and stores similar or related data and allows it to be retrieved as needed.

[0147] "Real-time" refers to a state in which data is generated or modified immediately and the results are reflected immediately to the user.

[0148] "Input means" refers to an interface or device that allows a user to input feature information and feedback to the system.

[0149] The "display means" refers to a screen or monitor that visually presents the generated portrait and the correction results to the user.

[0150] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[0151] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[0152] System Configuration

[0153] The system consists of three main components:

[0154] 1. User Interface (UI): The screen through which witnesses and users interact.

[0155] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0156] 3. Terminal: An input device used by witnesses and users.

[0157] The system operates as follows.

[0158] 1. Initial Setup

[0159] The terminal starts up and the user (police officer) logs in. The server verifies the login information and authentication is completed. The server establishes a connection to the database and initializes the caricature generation algorithm. A message indicating initial setup is complete is displayed on the terminal.

[0160] 2. Hearing from witnesses

[0161] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the information into the device's UI. The device then sends the input data to the server in real time.

[0162] 3. Caricature Generation

[0163] The server generates an initial portrait based on the received feature information, and then transmits the generated portrait data to the terminal, where the initial portrait is displayed.

[0164] 4. Adjustments and feedback

[0165] The user (witness) checks the sketch and tells the user (police officer) what corrections they would like to make (e.g., "Make the eyes thinner"). The user (police officer) enters the corrections they would like to make into the device's UI. The device sends the correction data to the server. The server updates the sketch based on the correction data, and the server sends the updated sketch data back to the device. The updated sketch is displayed on the device. This process is repeated until the witness is satisfied.

[0166] 5. Final confirmation and saving

[0167] The users (witness and police officer) review the final sketch, and if they are satisfied, they agree on it as the final sketch. The user (police officer) selects "Save." The server saves the final sketch data in the database. The server notifies the device that saving is complete, and a message indicating this is displayed on the device.

[0168] Specific examples

[0169] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0170] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0171] 2. The server generates the first portrait based on the feature information and sends it to the device.

[0172] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0173] 4. The user (police officer) inputs the correction, such as "the eyes should be narrower," and the device sends the correction information to the server.

[0174] 5. The server updates the portrait based on the correction data and sends it back to the device.

[0175] 6. Repeat this process until you have created a portrait that satisfies the witness.

[0176] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0177] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0178] Prompt Sentence Examples

[0179] markdown

[0180] This AI model is used to generate a portrait based on witness testimony, which includes the following:

[0181] Hair color: Black

[0182] Face Shape: Round

[0183] Eye shape: Narrow

[0184] Use these features to generate an initial caricature, then follow the instructions to make any necessary corrections.

[0185] In this way, this system uses artificial intelligence technology to quickly and accurately convert eyewitness testimony into a facial sketch, dramatically improving the efficiency of criminal investigations.

[0186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0187] Step 1: Initial Setup

[0188] The terminal boots up and the user accesses a login screen, which takes a username and password as input.

[0189] The server receives the username and password and verifies the authentication information using an SQL database. The authentication information is verified as part of the data processing, and the authentication result (success / failure) is obtained as the output.

[0190] If the server is successful in authentication, it establishes a connection to the database and initializes the caricature generation algorithm. The output is the connection status and the initialization completion status of the algorithm.

[0191] A message indicating that initial setup is complete is displayed on the terminal, and the user confirms that the system is ready.

[0192] Step 2: Interview witnesses

[0193] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and obtains characteristic information from the witness as input.

[0194] The user (police officer) inputs the feature information into the device's UI. The feature information is received as input data.

[0195] The terminal transmits the input characteristic information to the server in real time. The characteristic information is received as data processing, and the characteristic information is output and transmitted to the server.

[0196] Step 3: Generate a caricature

[0197] The server receives the feature information sent from the device and inputs it into the generative AI model, using the feature information as input data.

[0198] The server uses the generative AI model to generate an initial portrait based on the feature information. The generation process is carried out as a data calculation, and the generated portrait data is obtained as the output.

[0199] The server transmits the generated portrait data to the terminal.

[0200] The device displays the sketch on the screen, and the user (police officer) asks a witness to confirm the sketch.

[0201] Step 4: Adjustments and feedback

[0202] The user (witness) checks the generated sketch and tells the police officer what they would like to improve (e.g., "Make the eyes thinner"). Feedback information is received as input.

[0203] The user (police officer) inputs the desired corrections into the device's UI. The correction information is received as input data.

[0204] The terminal transmits the correction data to the server. The correction information is received as data processing, and the correction data is output and transmitted to the server.

[0205] The server re-executes the generative AI model based on the received correction data, and updates the portrait. The update process is performed as data calculation, and the updated portrait data is obtained as output.

[0206] The server transmits the updated portrait data to the terminal.

[0207] The device displays the updated sketch on the screen, and the user (police officer) asks the witness to review the updated sketch. This process is repeated until the witness is satisfied.

[0208] Step 5: Final review and save

[0209] The users (witnesses and police officers) review the final sketch and, if they are satisfied, agree on it as the final version.,Final confirmation information is received as input.

[0210] The user (police officer) selects "Save." A save instruction is received as an operational input.

[0211] The server saves the final portrait data in the database. The data saving process is performed as data processing, and the saving completion status is obtained as output.

[0212] The server notifies the terminal that the save is complete.

[0213] The terminal displays a message indicating that the data has been saved, and the process ends. The user (police officer) confirms that the process has been completed successfully.

[0214] In this way, through the specific actions and data processing performed at each step, the system can generate and modify highly accurate portraits in real time based on eyewitness testimony.

[0215] (Application example 1)

[0216] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0217] Traditional criminal investigation sketch creation processes are often manual, making it difficult to quickly and accurately generate sketches. Furthermore, due to a lack of a mechanism for incorporating real-time feedback from witnesses, it is difficult to quickly create a final sketch that satisfies the witnesses. This makes it difficult to effectively respond in situations where speed and accuracy are required in investigations.

[0218] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0219] In this invention, the server includes means for inputting feature information from a witness, means for generating a portrait based on the input feature information, means for displaying the generated portrait to the witness, means for inputting feedback from the witness, means for modifying the portrait based on the input feedback, means for displaying the modified portrait again to the witness, means for saving the final portrait in a database, means for transmitting and receiving feature information to the server in real time using a smart device, means for transmitting and receiving feedback to the server in real time, means for inputting feature information and feedback using prompt sentences, and means for generating and modifying a portrait based on the feature information and feedback using a generative AI model, thereby enabling rapid and highly accurate generation and modification of portraits.

[0220] "Characteristic information" is information provided by witnesses that indicates the criminal's external features and characteristics.

[0221] A "portrait" is a drawing that represents an image of the criminal's face, generated based on feature information.

[0222] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate and modify caricatures based on feature information and feedback.

[0223] A "prompt" is a phrase that guides witnesses and police officers when entering characteristic information and feedback.

[0224] "Feedback" is information indicating corrections or requests for improvement provided by a witness to the displayed portrait.

[0225] "Smart devices" refer to portable electronic devices with advanced functions, such as smartphones and tablets.

[0226] "Server" refers to a central processing unit that receives feature information and feedback, generates and modifies caricatures, and stores them in a database.

[0227] An "eyewitness" is someone who actually witnesses a crime scene or the circumstances of an incident and provides testimony.

[0228] The "database" is a data storage system for permanently storing the generated portraits and related information.

[0229] An "input device" is a hardware device used by a witness or police officer to input characteristic information or feedback.

[0230] This invention is a system that quickly generates a portrait based on characteristic information obtained from witnesses and makes corrections as necessary. This system generates a portrait in real time based on witness testimony, supporting investigations with high accuracy and efficiency.

[0231] System configuration

[0232] The system of the present invention comprises the following main components:

[0233] 1. User Interface (UI): A screen for interaction between witnesses and police officers. Implemented on smart devices (smartphones and tablets), it includes a testimony input form, a portrait display area, adjustment buttons, etc.

[0234] 2. Server: A central processing unit that processes caricature generation. It receives feature information and feedback, and generates and modifies caricatures using generative AI models.

[0235] 3. Database: A storage system for storing the generated portraits and related information.

[0236] Program processing overview

[0237] Hardware and software used

[0238] Hardware: Smart devices (smartphones, tablets), servers

[0239] Software: React Native or Swift UI (user interface), Flask (backend), TENSORFLOW® (generative AI model), MySQL® or MongoDB (database)

[0240] Data processing and calculation

[0241] Input and transmission of characteristic information: The user inputs the characteristic information of the witness through the UI on the smart device, and this information is transmitted to the server in real time.

[0242] Caricature generation: The server generates a caricature using TensorFlow based on the received feature information. The generated caricature is sent back to the smart device and displayed on the UI.

[0243] Feedback acquisition and revision: The witness enters feedback (requests for revision) on the portrait, which is then sent back to the server. The server then revises the portrait based on the feedback and sends the updated portrait back to the smart device. This process is repeated until the witness is satisfied.

[0244] Final saving: Once a final satisfactory caricature is generated, it is saved in the database.

[0245] Specific examples

[0246] For example, consider the case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." The user inputs characteristics such as "black hair," "round face," and "narrow eyes" into the UI of their smart device, which is then sent to the server. The server generates an initial sketch based on this characteristic information and sends it back to the smart device. If the witness provides feedback such as "his eyes are a little narrower," the sketch is sent again to the server, and the sketch is revised and reviewed by the witness. This process is repeated until the witness is satisfied, and a satisfactory sketch is generated and saved.

[0247] Prompt Sentence Examples

[0248] Text prompt:

[0249] TXT

[0250] Testimony input prompt:

[0251] Please enter the characteristics information provided by the witness in the following format:

[0252] Name: [Witness A]

[0253] Age: [Around 30 years old]

[0254] Hair color: [Black]

[0255] Eye shape: [narrow]

[0256] Once you have completed the input, please press the "Submit" button.

[0257] Caricature correction prompt:

[0258] Please review your current portrait and enter any necessary corrections in the following format:

[0259] Correction: [Eyes are a little narrower]

[0260] Reason for revision: [To make the impression of the actual perpetrator more realistic]

[0261] Once you have completed the entry, press the "Edit" button.

[0262] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0263] Step 1:

[0264] The terminal starts up and the officer logs in.

[0265] Input: Police officer's login information (username, password)

[0266] Operation: The device sends login information to the server. The server verifies the login information and performs authentication. If authentication is successful, the device displays a message indicating initial setup is complete.

[0267] Output: A notification that initial setup is complete will be displayed on the device.

[0268] Step 2:

[0269] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[0270] Input: Witness characteristics (e.g., age, gender, hair color, face shape, eye shape)

[0271] Operation: The device sends the input feature information to the server in real time.

[0272] Output: The feature information is sent to the server for processing.

[0273] Step 3:

[0274] The server generates an initial portrait based on the received feature information.

[0275] Input: feature information

[0276] How it works: The generative AI model on the server generates a portrait based on feature information.

[0277] Output: The generated portrait data is created and sent to the device.

[0278] Step 4:

[0279] The initial portrait is displayed on the device, and the user (witness) checks it.

[0280] Input: Generated caricature data

[0281] How it works: The device displays a sketch to a witness, who reviews it and tells the police officer what corrections they would like to make.

[0282] Output: The witness' desired correction score is obtained.

[0283] Step 5:

[0284] The user (police officer) inputs the desired corrections (e.g., "the eyes should be narrower") into the device's UI.

[0285] Input: Witness' desired correction point

[0286] Operation: The device sends the desired corrections to the server.

[0287] Output: The desired correction points are sent to the server for processing.

[0288] Step 6:

[0289] The server updates the portrait based on the desired corrections.

[0290] Input: desired corrections

[0291] How it works: The generative AI model on the server modifies the portrait, taking into account the desired corrections.

[0292] Output: The corrected portrait data is created and sent to the device.

[0293] Step 7:

[0294] The updated portrait is displayed on the device, and the user (witness) checks it again.

[0295] Input: Corrected portrait data

[0296] How it works: The device displays the revised portrait to the witness. If the witness is not satisfied, the process returns to input further revisions.

[0297] Output: The witness's desired correction points are obtained, or the final portrait is confirmed by the witness.

[0298] Step 8:

[0299] The user (police officer) checks the final portrait and selects "Save" if they are satisfied.

[0300] Input: Final caricature approval

[0301] Operation: The device sends a "save" request to the server, which saves the final portrait data to a database.

[0302] Output: A notification that the final portrait has been saved will be displayed on your device.

[0303] By following these processing steps, a system is realized that can quickly and accurately generate and modify caricatures in real time.

[0304] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0305] This invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on characteristic information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[0306] This system includes a series of processes for acquiring feature information from witnesses and generating and modifying caricatures, as well as a function for recognizing the emotions of witnesses and dynamically adjusting the interface and generation process accordingly.The system configuration and its specific program processing are explained below.

[0307] System Configuration

[0308] The system consists of four main components:

[0309] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[0310] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0311] 3. Terminal: An input device used by witnesses and police officers.

[0312] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[0313] Program processing overview

[0314] 1. Initial Setup

[0315] The terminal starts up and the officer enters their login information.

[0316] The server verifies the login information and authentication is complete.

[0317] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0318] A message will appear on the device indicating that the initial setup is complete.

[0319] 2. Hearing from witnesses

[0320] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0321] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[0322] The terminal transmits input data to the server in real time.

[0323] 3. Eyewitness emotion recognition

[0324] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[0325] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[0326] 4. Caricature Generation

[0327] The server generates an initial portrait based on the received feature information.

[0328] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[0329] 5. Adjustments and Feedback

[0330] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0331] The user (police officer) inputs the desired corrections into the device's UI.

[0332] The terminal transmits the correction data to the server.

[0333] The server updates the portrait based on the correction data and sends it back to the device.

[0334] The corrected portrait will be displayed on the device, and this process will be repeated until the witness is satisfied.

[0335] 6. Final confirmation and saving

[0336] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[0337] The user (police officer) selects "Save."

[0338] The server stores the final portrait data in a database.

[0339] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[0340] Specific examples

[0341] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0342] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0343] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[0344] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[0345] 4. The server generates the first portrait based on the feature information and sends it to the device.

[0346] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0347] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[0348] 7. The server updates the portrait based on the correction data and sends it back to the device.

[0349] 8. Repeat this process until you have created a portrait that satisfies the witness.

[0350] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0351] The system allows witnesses to provide feedback in a relaxed manner, improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0352] The processing flow will be explained below.

[0353] Step 1:

[0354] The terminal starts up and the officer enters their login information.

[0355] Step 2:

[0356] The server verifies the login information and authentication is complete.

[0357] Step 3:

[0358] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0359] Step 4:

[0360] A message will appear on the device indicating that the initial setup is complete.

[0361] Step 5:

[0362] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0363] Step 6:

[0364] The user (police officer) inputs the obtained characteristic information into the terminal's UI.

[0365] Step 7:

[0366] The terminal transmits input data to the server in real time.

[0367] Step 8:

[0368] The server generates an initial portrait based on the received feature information.

[0369] Step 9:

[0370] The server transmits the generated portrait data to the terminal.

[0371] Step 10:

[0372] The first portrait will be displayed on the device.

[0373] Step 11:

[0374] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[0375] Step 12:

[0376] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[0377] Step 13:

[0378] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0379] Step 14:

[0380] The user (police officer) inputs the desired corrections into the device's UI.

[0381] Step 15:

[0382] The terminal transmits the correction data to the server.

[0383] Step 16:

[0384] The server updates the portrait based on the correction data.

[0385] Step 17:

[0386] The server transmits the updated portrait data to the terminal again.

[0387] Step 18:

[0388] The revised portrait will be displayed on the device.

[0389] Step 19:

[0390] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[0391] Step 20:

[0392] The users (witnesses and police officers) agree on the final sketch.

[0393] Step 21:

[0394] The user (police officer) selects "Save."

[0395] Step 22:

[0396] The server stores the final portrait data in a database.

[0397] Step 23:

[0398] The server notifies the terminal that the save is complete.

[0399] Step 24:

[0400] A message will appear on your device indicating that the save is complete.

[0401] Example 2

[0402] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0403] In conventional portrait generation systems, the input of information from witnesses and the correction of portraits are often cumbersome and time-consuming. Furthermore, witness tension and anxiety can affect the accuracy of the portrait. For this reason, there is a need to create an environment where witnesses can relax and provide detailed feedback. Furthermore, because it is difficult to generate and correct portraits in real time, there is a need for highly accurate and rapid investigative support.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0405] In this invention, the server includes means for recognizing emotions from the facial expressions and voices of witnesses, means for dynamically adjusting a user interface based on the recognized emotion information, means for generating a caricature based on input feature information, and means for modifying the caricature based on input feedback. This allows the witness to provide feedback in a relaxed state, and enables the generation and modification of highly accurate caricatures in real time.

[0406] "Means for inputting characteristic information from witnesses" refers to an interface or device that allows witnesses to provide information about the appearance and characteristics of the perpetrator to the system.

[0407] "Means for generating a caricature based on input feature information" refers to an algorithm or program for automatically creating a caricature based on appearance and features input by a witness.

[0408] "Means for displaying the generated caricature to a witness" refers to an interface or device for showing the generated caricature to a witness through a display or monitor.

[0409] "Means for inputting feedback from witnesses" refers to an interface or device that allows witnesses to input their opinions and comments to the system for corrections and improvements to the generated portrait.

[0410] "Means for modifying the portrait based on input feedback" refers to an algorithm or program that automatically modifies the portrait based on feedback provided by witnesses.

[0411] "Means for displaying the modified portrait to the witness again" refers to an interface or device for displaying the modified portrait to the witness again through a display or monitor.

[0412] "Means for storing the final portrait in a database" refers to a system or program for storing the finalized portrait in a database in digital format.

[0413] "Means for recognizing emotions from the facial expressions and voice of witnesses" refers to emotion recognition algorithms or programs that analyze the facial expressions and voice of witnesses to identify their emotional state (e.g., tension, anxiety, relaxation) in real time.

[0414] "Means for dynamically adjusting a user interface based on recognized emotional information" refers to a system or program for automatically modifying or adjusting an interface in response to a recognized emotional state of a witness to allow the user to relax and provide feedback.

[0415] "Server" refers to a computer system that is a central processing unit for executing each of the above means and is responsible for major processes such as receiving, transmitting, and storing data, as well as generating and modifying portraits.

[0416] "Device" refers to the hardware device used by witnesses and police officers to input characteristic information and feedback and to display the sketch, including, for example, a computer, tablet, or smartphone.

[0417] An "emotion engine" refers to software or algorithms that analyze facial expressions and voice of witnesses to recognize their emotions and dynamically adjust the user interface or system behavior based on those emotions.

[0418] "User Interface (UI)" refers to the screen displays and controls used by witnesses and police officers to interact with the system, including the design and functionality to facilitate inputting information and providing feedback.

[0419] The present invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on feature information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[0420] System Configuration

[0421] The system consists of four main components:

[0422] 1. User Interface (UI): The screen through which witnesses and investigators interact.

[0423] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0424] 3. Terminal: An input device used by witnesses and police officers.

[0425] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[0426] Program processing

[0427] The server includes a means for inputting feature information from a witness, a means for generating a caricature based on the input feature information, a means for displaying the generated caricature to the witness, a means for inputting feedback from the witness, a means for modifying the caricature based on the feedback, a means for displaying the modified caricature again to the witness, and a means for saving the final caricature in a database. The system also includes an emotion engine for recognizing emotions from the facial expressions and voices of the witness, and a means for dynamically adjusting the UI based on the recognition results.

[0428] Hardware and Software

[0429] Hardware: The system components include a PC, a tablet, and a networked server.

[0430] Software: Includes emotion recognition algorithms, similar face generation algorithms, and database management systems.

[0431] Specific examples

[0432] As an example of implementation, we will explain a case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." Here, we will specifically explain the input process, emotion recognition process, caricature generation process, and correction process.

[0433] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0434] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[0435] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[0436] 4. The server generates the first portrait based on the feature information and sends it to the device.

[0437] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0438] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[0439] 7. The server updates the portrait based on the correction data and sends it back to the device.

[0440] 8. Repeat this process until you have created a portrait that satisfies the witness.

[0441] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0442] Prompt Sentence Examples

[0443] Below are some examples of specific prompts that can be input to the generative AI model in this system:

[0444] "A witness listed the suspect's characteristics as 'black hair, a round face, and narrow eyes.' Please generate an initial sketch based on this information."

[0445] "We have detected the witness' emotional state. They do not appear to be relaxed. Please adjust the user interface to help them relax."

[0446] "A witness pointed out a correction to the portrait, saying 'the eyes are narrower.' Please generate a new portrait that reflects this."

[0447] This invention allows witnesses to provide feedback in a relaxed state, improving the speed and accuracy of investigations and reducing the risk of false accusations.

[0448] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0449] Program processing flow

[0450] Step 1: Initial Setup

[0451] Step 1-1: Start the terminal and log in

[0452] Input: Police officer's login information (ID and password)

[0453] Output: Server authentication result

[0454] Specific operation:

[0455] The device will start up and display the login screen.

[0456] The user (police officer) enters their ID and password and clicks the "Login" button.

[0457] The terminal sends the input information to the server, which verifies the login information.

[0458] The server sends the authentication result to the terminal, and the terminal displays a message indicating "authentication successful" or "authentication failed."

[0459] Step 1-2: Establishing a database connection

[0460] Input: Server authentication success result

[0461] Output: Database connection established message

[0462] Specific operation:

[0463] Once the server receives the successful authentication message, it attempts to connect to the database.

[0464] The server initializes the caricature generation algorithm.

[0465] The server sends a message to the terminal indicating that the database connection has been successful, and the terminal displays the message "Initial settings completed."

[0466] Step 2: Hear from witnesses

[0467] Step 2-1: Interviewing witnesses

[0468] Input: Basic information of the witness (e.g., age, gender, hair color, etc.)

[0469] Output: Input data of eyewitness characteristics

[0470] Specific operation:

[0471] The user (police officer) asks witnesses questions about the basic characteristics of the perpetrator.

[0472] The user (police officer) enters the witness's answers into the device's UI.

[0473] Step 2-2: Real-time transmission of feature information

[0474] Input: Witness characteristics information entered into the terminal

[0475] Output: Feature information data sent to the server

[0476] Specific operation:

[0477] The device transmits the entered characteristic information of the witness to the server in real time.

[0478] The server receives the characteristic information and temporarily stores it in a database.

[0479] Step 3: Eyewitness emotion recognition

[0480] Step 3-1: Acquiring emotion data

[0481] Input: witness' facial expressions and voice data

[0482] Output: Recognized emotion information

[0483] Specific operation:

[0484] The emotion engine scans the facial expressions and voices of witnesses in real time via the device.

[0485] The emotion engine analyzes the collected data to recognize the witness's emotions (e.g., nervous, anxious, relaxed).

[0486] The emotion engine sends the recognition results to the server.

[0487] Step 3-2: Dynamic UI adjustment

[0488] Input: Recognized emotion information

[0489] Output: Adjusted user interface

[0490] Specific operation:

[0491] The server analyzes emotional information and determines UI settings that are easy to relax.

[0492] The server sends instructions to the terminal to change the UI settings, and the terminal dynamically adjusts the UI.

[0493] Step 4: Generate a caricature

[0494] Step 4-1: Generate the first portrait

[0495] Input: Witness characteristics information

[0496] Output: First portrait data

[0497] Specific operation:

[0498] The server runs an algorithm to generate an initial portrait based on the feature information.

[0499] The server transmits the generated portrait data to the terminal.

[0500] Step 4-2: Displaying the caricature

[0501] Input: First portrait data

[0502] Output: Initial portrait shown to witnesses

[0503] Specific operation:

[0504] The first portrait received by the device is displayed on the screen.

[0505] Step 5: Adjustments and feedback

[0506] Step 5-1: Input desired corrections

[0507] Input: Eyewitness feedback

[0508] Output: Data of desired correction points

[0509] Specific operation:

[0510] The user (witness) checks the sketch and tells the user (police officer) what needs to be corrected.

[0511] The user (police officer) inputs the desired corrections into the UI.

[0512] Step 5-2: Submitting the correction data

[0513] Input: Correction requests entered into the UI

[0514] Output: Corrected data sent to the server

[0515] Specific operation:

[0516] The terminal transmits the desired corrections to the server.

[0517] Step 5-3: Update your portrait

[0518] Input: Data of desired correction points

[0519] Output: Updated caricature data

[0520] Specific operation:

[0521] The server runs an algorithm to update the portrait based on the correction data.

[0522] The server transmits the updated portrait data to the terminal again.

[0523] Step 5-4: Displaying the updated caricature

[0524] Input: Updated caricature data

[0525] Output: Updated portrait displayed to witnesses

[0526] Specific operation:

[0527] The updated portrait received by the terminal is displayed on the display.

[0528] This process is repeated until the witness is satisfied.

[0529] Step 6: Final review and save

[0530] Step 6-1: Final confirmation

[0531] Input: Eyewitness confirmation of updated portrait

[0532] Output: Final caricature data

[0533] Specific operation:

[0534] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[0535] Step 6-2: Save the portrait

[0536] Input: Final portrait data

[0537] Output: Caricature data stored in a database

[0538] Specific operation:

[0539] The user (police officer) selects "Save."

[0540] The server stores the final portrait data in a database.

[0541] Step 6-3: Notification of save completion

[0542] Input: Database save completion result

[0543] Output: Message notifying you that saving is complete

[0544] Specific operation:

[0545] The server notifies the terminal that the save is complete, and the message "Saving complete" is displayed on the terminal.

[0546] (Application example 2)

[0547] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0548] In criminal investigations, conventional methods of generating sketches based on eyewitness testimony can lead to inaccurate testimony due to the eyewitness's emotions or nervousness, and can hinder smooth communication with investigators. Furthermore, generating and modifying sketches in real time is difficult, resulting in a lack of speed and accuracy in investigations. To solve these issues, a system is needed that can recognize eyewitness emotions and dynamically adjust the interface to generate and modify sketches.

[0549] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0550] In this invention, the server includes means for recognizing emotions from the facial expressions and voice of a witness, means for dynamically adjusting the interface based on the recognized emotion information, and means for generating a portrait based on the input feature information. This allows the witness to give accurate testimony in a relaxed state, and enables rapid and highly accurate generation of a portrait.

[0551] A "witness" is someone who directly witnessed an incident or accident, or was involved in it face-to-face.

[0552] "Characteristic information" refers to information that records details of the appearance and behavior of a person described by a witness.

[0553] A "caricature" is an illustration or image of a person's face based on eyewitness testimony.

[0554] "Facial expression" refers to temporary emotions and reactions expressed through facial muscle movements, gaze, etc.

[0555] "Voice" refers to the linguistic expressions such as tone, intonation, and volume of the voice spoken by the witness.

[0556] "Emotional information" refers to data that represents the psychological state of a witness analyzed by the emotion engine.

[0557] "Interface" refers to the screen and operating means for exchanging information between the user and the system.

[0558] "Feedback" refers to the behavior of witnesses expressing their opinions or requests for corrections regarding the generated portrait.

[0559] "Server" refers to a remote computer for storing, processing, and transmitting data.

[0560] This invention is a system that acquires feature information based on eyewitness testimony and uses an emotion engine to generate and modify highly accurate caricatures in real time. The system takes into account the emotions of the eyewitness and modifies the caricature as many times as necessary until the eyewitness is satisfied, and finally saves it in a database.

[0561] System programs and the hardware and software used

[0562] The system consists of the following main components:

[0563] 1. Server: Generates and modifies caricatures based on feature and emotion information.

[0564] Software used: Emotion recognition software, caricature generation algorithm

[0565] Hardware used: high-speed processor, database server

[0566] 2. Terminal: An input device used by witnesses or investigators to transmit input feature information and feedback information to the server. Examples include smart glasses, smartphones, and tablets.

[0567] Software used: Input interface, emotion recognition software

[0568] Hardware used: Smart glasses (e.g., Google® Glass®, Vuzix Blade), smartphone (iOS / ANDROID®)

[0569] 3. User Interface (UI): A screen for witness and investigator interaction, displaying the sketch generated in real time and providing the ability to input witness feedback.

[0570] Software used: Browser-based UI, desktop application, mobile app

[0571] Explanation of program processing

[0572] The server uses emotion recognition software to recognize emotions from the witness's facial expressions and voice. This software analyzes the witness's facial expression data and voice data to recognize their current emotional state (e.g., nervous, anxious, relaxed). Based on the emotional information, the server dynamically adjusts the interface to provide an environment in which the witness can relax and easily testify.

[0573] When feature information is entered on the device, it is sent to the server, which then generates an initial caricature based on the feature information using a caricature generation algorithm. The generated caricature is then displayed to the witness through a user interface. When the witness provides feedback, the device sends the information to the server, which then modifies the caricature based on the feedback.

[0574] This process is repeated until the witness is satisfied, and once the final portrait is completed, the server stores it in a database.

[0575] Specific examples

[0576] For example, if a witness says, "The perpetrator has black hair, a round face, and narrow eyes," these characteristics are entered into the device. The server uses emotion recognition software to analyze whether the witness is nervous or relaxed. If it detects nervousness, the server adjusts the user interface, changing the interface design and the questions presented to put the witness at ease.

[0577] After the first sketch is generated and displayed on the device, the witness may point out that the eyes are narrower. This feedback is sent to the server, which then modifies the sketch based on the feedback. This process is repeated several times until a final sketch is created that satisfies the witness.

[0578] Prompt Sentence Examples

[0579] "Please tell us the characteristics of the perpetrator. Please be as detailed as possible. What is the shape of the eyes? What color is the hair? What is the shape of the face? Please relax and answer."

[0580] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0581] Step 1:

[0582] The server initializes the system. The terminal starts up and the investigator enters their login information. Once the login information is entered, the server verifies it and performs authentication. Once authentication is complete, the server establishes a connection to the database and initializes the portrait generation algorithm. At this point, the terminal displays a message indicating that initialization is complete.

[0583] Input: Investigator login information

[0584] Output: Authentication complete message, initial setup complete message

[0585] Data processing / calculation: Authenticating login information, establishing database connections, initializing algorithms

[0586] Step 2:

[0587] The user (investigator) asks the witness for basic information (e.g., age, gender, hair color, etc.) in an interview format. The basic information provided by the witness is entered into the device's UI. The device then transmits the entered characteristic information to the server in real time.

[0588] Input: basic information and characteristics of the witness

[0589] Output: Sending the input feature information

[0590] Data processing / calculation: Input and organization of basic information, real-time transmission of characteristic information

[0591] Step 3:

[0592] The server receives the feature information and uses an emotion engine to recognize emotions from the witness's facial expressions and voice. The recognized emotion information is analyzed on the server side, and the UI is dynamically adjusted to make the witness feel relaxed and easy to give feedback.

[0593] Input: witness's facial expression data, voice data

[0594] Output: Recognized emotion information

[0595] Data processing / calculation: Analysis of facial expression data and voice data, emotion recognition, dynamic UI adjustment

[0596] Step 4:

[0597] The server generates an initial portrait based on the received feature information. A portrait generation algorithm is used for this generation. The generated portrait data is sent to the terminal, and the initial portrait is displayed on the terminal.

[0598] Input: feature information

[0599] Output: Sending and displaying the initial portrait data

[0600] Data processing / calculation: Generate portraits based on feature information, send portrait data

[0601] Step 5:

[0602] The user (witness) checks the sketch and tells the investigator what they would like to be corrected (e.g., "Make the eyes thinner"). The investigator inputs the witness's feedback into the device's UI. The input correction requests are sent from the device to the server.

[0603] Input: Witness feedback, correction requests

[0604] Output: Sending correction requests

[0605] Data processing / calculation: Input and sorting of feedback, sending correction requests

[0606] Step 6:

[0607] The server then modifies the portrait based on the requested corrections. The modified portrait data is then sent back to the terminal and displayed on the device. This process is repeated until the witness is satisfied.

[0608] Input: desired corrections

[0609] Output: Send and display the corrected portrait data

[0610] Data processing / calculation: Correction of portraits based on desired correction points, resending of portrait data

[0611] Step 7:

[0612] The users (witness and investigator) review the final sketch and, if satisfied, agree on it as the final sketch. The investigator selects "Save," and the server saves the final sketch data in the database. A notification that saving is complete is displayed on the device.

[0613] Input: Final portrait data

[0614] Output: Database save, save completion notification

[0615] Data processing / calculation: Saving portrait data and sending a notification when saving is complete

[0616] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0617] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0618] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0619] [Second embodiment]

[0620] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0621] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0622] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0623] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0624] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0625] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0626] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0627] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0628] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0629] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0630] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0631] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0632] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[0633] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[0634] System Configuration

[0635] The system consists of three main components:

[0636] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[0637] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0638] 3. Terminal: An input device used by witnesses and police officers.

[0639] Program processing overview

[0640] 1. Initial Setup

[0641] The terminal starts up and the officer logs in.

[0642] The server verifies the login information and authentication is complete.

[0643] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0644] A message will appear on the device indicating that the initial setup is complete.

[0645] 2. Hearing from witnesses

[0646] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[0647] The terminal transmits input data to the server in real time.

[0648] 3. Caricature Generation

[0649] The server generates an initial portrait based on the received feature information.

[0650] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[0651] 4. Adjustments and feedback

[0652] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0653] The user (police officer) inputs the desired corrections into the device's UI.

[0654] The terminal transmits the correction data to the server.

[0655] The server updates the portrait based on the correction data, and transmits the updated portrait data to the terminal again.

[0656] The updated portrait will be displayed on the device, and the process will be repeated until the witness is satisfied.

[0657] 5. Final confirmation and saving

[0658] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[0659] The user (police officer) selects "Save."

[0660] The server stores the final portrait data in a database.

[0661] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[0662] Specific examples

[0663] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0664] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0665] 2. The server generates the first portrait based on the feature information and sends it to the device.

[0666] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0667] 4. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[0668] 5. The server updates the portrait based on the correction data and sends it back to the device.

[0669] 6. Repeat this process until you have created a portrait that satisfies the witness.

[0670] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0671] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0672] The processing flow will be explained below.

[0673] Step 1:

[0674] The terminal starts up and the officer enters their login information.

[0675] Step 2:

[0676] The server verifies the login information and authentication is complete.

[0677] Step 3:

[0678] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0679] Step 4:

[0680] A message will appear on the device indicating that the initial setup is complete.

[0681] Step 5:

[0682] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0683] Step 6:

[0684] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[0685] Step 7:

[0686] The terminal transmits the input characteristic information to the server in real time.

[0687] Step 8:

[0688] The server generates an initial portrait based on the received feature information.

[0689] Step 9:

[0690] The server transmits the generated portrait data to the terminal.

[0691] Step 10:

[0692] The first portrait will be displayed on the device.

[0693] Step 11:

[0694] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0695] Step 12:

[0696] The user (police officer) inputs the desired corrections into the device's UI.

[0697] Step 13:

[0698] The terminal transmits the correction data to the server.

[0699] Step 14:

[0700] The server updates the portrait based on the correction data.

[0701] Step 15:

[0702] The server transmits the updated portrait data to the terminal again.

[0703] Step 16:

[0704] The revised portrait will be displayed on the device.

[0705] Step 17:

[0706] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[0707] Step 18:

[0708] The users (witnesses and police officers) agree on the final sketch.

[0709] Step 19:

[0710] The user (police officer) selects "Save."

[0711] Step 20:

[0712] The server stores the final portrait data in a database.

[0713] Step 21:

[0714] The server notifies the terminal that the save is complete.

[0715] Step 22:

[0716] A message will appear on your device indicating that the save is complete.

[0717] Example 1

[0718] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] In criminal investigations, the task of creating a portrait based on eyewitness testimony is time-consuming and labor-intensive. Furthermore, eyewitnesses' memories are often vague, requiring repeated revisions, making it difficult to quickly generate a highly accurate portrait. To address these challenges, a technology is needed that can generate and revise portraits in real time based on information obtained from eyewitnesses, ultimately resulting in a highly accurate portrait.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0721] In this invention, the server includes a means for a user to input feature information from a witness, a means for generating a portrait using a generative AI model based on the input feature information, and a means for displaying the corrected portrait to the user again. This allows for the generation and correction of portraits based on witness testimony in real time, thereby improving the efficiency and accuracy of investigations.

[0722] The "user" is the person who operates the system, inputs information from witnesses, and reviews the generated and modified portraits.

[0723] A "witness" is someone who saw a suspect at a crime scene or elsewhere, and whose testimony is used to generate or modify the sketch.

[0724] "Characteristic information" refers to specific physical characteristics of a suspect as testified by witnesses (e.g., age, gender, hair color, face shape, eye shape, etc.).

[0725] The "server" is the central computing device of the system, a combination of hardware and software that receives feature information, generates and modifies portraits using generative AI models, and stores data.

[0726] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate and modify caricatures in real time using feature information as input.

[0727] A "caricature" is an image of a suspect's face generated from witness testimony and can be digitally displayed and modified.

[0728] "Feedback" is information including desired corrections and suggestions made by the witness to the generated portrait.

[0729] A "database" is a digital storage system that organizes and stores similar or related data and allows it to be retrieved as needed.

[0730] "Real-time" refers to a state in which data is generated or modified immediately and the results are reflected immediately to the user.

[0731] "Input means" refers to an interface or device that allows a user to input feature information and feedback to the system.

[0732] The "display means" refers to a screen or monitor that visually presents the generated portrait and the correction results to the user.

[0733] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[0734] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[0735] System Configuration

[0736] The system consists of three main components:

[0737] 1. User Interface (UI): The screen through which witnesses and users interact.

[0738] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0739] 3. Terminal: An input device used by witnesses and users.

[0740] The system operates as follows.

[0741] 1. Initial Setup

[0742] The terminal starts up and the user (police officer) logs in. The server verifies the login information and authentication is completed. The server establishes a connection to the database and initializes the caricature generation algorithm. A message indicating initial setup is complete is displayed on the terminal.

[0743] 2. Hearing from witnesses

[0744] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the information into the device's UI. The device then sends the input data to the server in real time.

[0745] 3. Caricature Generation

[0746] The server generates an initial portrait based on the received feature information, and then transmits the generated portrait data to the terminal, where the initial portrait is displayed.

[0747] 4. Adjustments and feedback

[0748] The user (witness) checks the sketch and tells the user (police officer) what corrections they would like to make (e.g., "Make the eyes thinner"). The user (police officer) enters the corrections they would like to make into the device's UI. The device sends the correction data to the server. The server updates the sketch based on the correction data, and the server sends the updated sketch data back to the device. The updated sketch is displayed on the device. This process is repeated until the witness is satisfied.

[0749] 5. Final confirmation and saving

[0750] The users (witness and police officer) review the final sketch, and if they are satisfied, they agree on it as the final sketch. The user (police officer) selects "Save." The server saves the final sketch data in the database. The server notifies the device that saving is complete, and a message indicating this is displayed on the device.

[0751] Specific examples

[0752] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0753] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0754] 2. The server generates the first portrait based on the feature information and sends it to the device.

[0755] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0756] 4. The user (police officer) inputs the correction, such as "the eyes should be narrower," and the device sends the correction information to the server.

[0757] 5. The server updates the portrait based on the correction data and sends it back to the device.

[0758] 6. Repeat this process until you have created a portrait that satisfies the witness.

[0759] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0760] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0761] Prompt Sentence Examples

[0762] markdown

[0763] This AI model is used to generate a portrait based on witness testimony, which includes the following:

[0764] Hair color: Black

[0765] Face Shape: Round

[0766] Eye shape: Narrow

[0767] Use these features to generate an initial caricature, then follow the instructions to make any necessary corrections.

[0768] In this way, this system uses artificial intelligence technology to quickly and accurately convert eyewitness testimony into a facial sketch, dramatically improving the efficiency of criminal investigations.

[0769] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0770] Step 1: Initial Setup

[0771] The terminal boots up and the user accesses a login screen, which takes a username and password as input.

[0772] The server receives the username and password and verifies the authentication information using an SQL database. The authentication information is verified as part of the data processing, and the authentication result (success / failure) is obtained as the output.

[0773] If the server is successful in authentication, it establishes a connection to the database and initializes the caricature generation algorithm. The output is the connection status and the initialization completion status of the algorithm.

[0774] A message indicating that initial setup is complete is displayed on the terminal, and the user confirms that the system is ready.

[0775] Step 2: Interview witnesses

[0776] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and obtains characteristic information from the witness as input.

[0777] The user (police officer) inputs the feature information into the device's UI. The feature information is received as input data.

[0778] The terminal transmits the input characteristic information to the server in real time. The characteristic information is received as data processing, and the characteristic information is output and transmitted to the server.

[0779] Step 3: Generate a caricature

[0780] The server receives the feature information sent from the device and inputs it into the generative AI model, using the feature information as input data.

[0781] The server uses the generative AI model to generate an initial portrait based on the feature information. The generation process is carried out as a data calculation, and the generated portrait data is obtained as the output.

[0782] The server transmits the generated portrait data to the terminal.

[0783] The device displays the sketch on the screen, and the user (police officer) asks a witness to confirm the sketch.

[0784] Step 4: Adjustments and feedback

[0785] The user (witness) checks the generated sketch and tells the police officer what they would like to improve (e.g., "Make the eyes thinner"). Feedback information is received as input.

[0786] The user (police officer) inputs the desired corrections into the device's UI. The correction information is received as input data.

[0787] The terminal transmits the correction data to the server. The correction information is received as data processing, and the correction data is output and transmitted to the server.

[0788] The server re-executes the generative AI model based on the received correction data, and updates the portrait. The update process is performed as data calculation, and the updated portrait data is obtained as output.

[0789] The server transmits the updated portrait data to the terminal.

[0790] The device displays the updated sketch on the screen, and the user (police officer) asks the witness to review the updated sketch. This process is repeated until the witness is satisfied.

[0791] Step 5: Final review and save

[0792] The users (witnesses and police officers) review the final sketch and, if they are satisfied, agree on it as the final version.,Final confirmation information is received as input.

[0793] The user (police officer) selects "Save." A save instruction is received as an operational input.

[0794] The server saves the final portrait data in the database. The data saving process is performed as data processing, and the saving completion status is obtained as output.

[0795] The server notifies the terminal that the save is complete.

[0796] The terminal displays a message indicating that the data has been saved, and the process ends. The user (police officer) confirms that the process has been completed successfully.

[0797] In this way, through the specific actions and data processing performed at each step, the system can generate and modify highly accurate portraits in real time based on eyewitness testimony.

[0798] (Application example 1)

[0799] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] Traditional criminal investigation sketch creation processes are often manual, making it difficult to quickly and accurately generate sketches. Furthermore, due to a lack of a mechanism for incorporating real-time feedback from witnesses, it is difficult to quickly create a final sketch that satisfies the witnesses. This makes it difficult to effectively respond in situations where speed and accuracy are required in investigations.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0802] In this invention, the server includes means for inputting feature information from a witness, means for generating a portrait based on the input feature information, means for displaying the generated portrait to the witness, means for inputting feedback from the witness, means for modifying the portrait based on the input feedback, means for displaying the modified portrait again to the witness, means for saving the final portrait in a database, means for transmitting and receiving feature information to the server in real time using a smart device, means for transmitting and receiving feedback to the server in real time, means for inputting feature information and feedback using prompt sentences, and means for generating and modifying a portrait based on the feature information and feedback using a generative AI model, thereby enabling rapid and highly accurate generation and modification of portraits.

[0803] "Characteristic information" is information provided by witnesses that indicates the criminal's external features and characteristics.

[0804] A "portrait" is a drawing that represents an image of the criminal's face, generated based on feature information.

[0805] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate and modify caricatures based on feature information and feedback.

[0806] A "prompt" is a phrase that guides witnesses and police officers when entering characteristic information and feedback.

[0807] "Feedback" is information indicating corrections or requests for improvement provided by a witness to the displayed portrait.

[0808] "Smart devices" refer to portable electronic devices with advanced functions, such as smartphones and tablets.

[0809] "Server" refers to a central processing unit that receives feature information and feedback, generates and modifies caricatures, and stores them in a database.

[0810] An "eyewitness" is someone who actually witnesses a crime scene or the circumstances of an incident and provides testimony.

[0811] The "database" is a data storage system for permanently storing the generated portraits and related information.

[0812] An "input device" is a hardware device used by a witness or police officer to input characteristic information or feedback.

[0813] This invention is a system that quickly generates a portrait based on characteristic information obtained from witnesses and makes corrections as necessary. This system generates a portrait in real time based on witness testimony, supporting investigations with high accuracy and efficiency.

[0814] System configuration

[0815] The system of the present invention comprises the following main components:

[0816] 1. User Interface (UI): A screen for interaction between witnesses and police officers. Implemented on smart devices (smartphones and tablets), it includes a testimony input form, a portrait display area, adjustment buttons, etc.

[0817] 2. Server: A central processing unit that processes caricature generation. It receives feature information and feedback, and generates and modifies caricatures using generative AI models.

[0818] 3. Database: A storage system for storing the generated portraits and related information.

[0819] Program processing overview

[0820] Hardware and software used

[0821] Hardware: Smart devices (smartphones, tablets), servers

[0822] Software: React Native or Swift UI (user interface), Flask (backend), TensorFlow (generative AI models), MySQL or MongoDB (database)

[0823] Data processing and calculation

[0824] Input and transmission of characteristic information: The user inputs the characteristic information of the witness through the UI on the smart device, and this information is transmitted to the server in real time.

[0825] Caricature generation: The server generates a caricature using TensorFlow based on the received feature information. The generated caricature is sent back to the smart device and displayed on the UI.

[0826] Feedback acquisition and revision: The witness enters feedback (requests for revision) on the portrait, which is then sent back to the server. The server then revises the portrait based on the feedback and sends the updated portrait back to the smart device. This process is repeated until the witness is satisfied.

[0827] Final saving: Once a final satisfactory caricature is generated, it is saved in the database.

[0828] Specific examples

[0829] For example, consider the case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." The user inputs characteristics such as "black hair," "round face," and "narrow eyes" into the UI of their smart device, which is then sent to the server. The server generates an initial sketch based on this characteristic information and sends it back to the smart device. If the witness provides feedback such as "his eyes are a little narrower," the sketch is sent again to the server, and the sketch is revised and reviewed by the witness. This process is repeated until the witness is satisfied, and a satisfactory sketch is generated and saved.

[0830] Prompt Sentence Examples

[0831] Text prompt:

[0832] TXT

[0833] Testimony input prompt:

[0834] Please enter the characteristics information provided by the witness in the following format:

[0835] Name: [Witness A]

[0836] Age: [Around 30 years old]

[0837] Hair color: [Black]

[0838] Eye shape: [narrow]

[0839] Once you have completed the input, please press the "Submit" button.

[0840] Caricature correction prompt:

[0841] Please review your current portrait and enter any necessary corrections in the following format:

[0842] Correction: [Eyes are a little narrower]

[0843] Reason for revision: [To make the impression of the actual perpetrator more realistic]

[0844] Once you have completed the entry, press the "Edit" button.

[0845] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0846] Step 1:

[0847] The terminal starts up and the officer logs in.

[0848] Input: Police officer's login information (username, password)

[0849] Operation: The device sends login information to the server. The server verifies the login information and performs authentication. If authentication is successful, the device displays a message indicating initial setup is complete.

[0850] Output: A notification that initial setup is complete will be displayed on the device.

[0851] Step 2:

[0852] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[0853] Input: Witness characteristics (e.g., age, gender, hair color, face shape, eye shape)

[0854] Operation: The device sends the input feature information to the server in real time.

[0855] Output: The feature information is sent to the server for processing.

[0856] Step 3:

[0857] The server generates an initial portrait based on the received feature information.

[0858] Input: feature information

[0859] How it works: The generative AI model on the server generates a portrait based on feature information.

[0860] Output: The generated portrait data is created and sent to the device.

[0861] Step 4:

[0862] The initial portrait is displayed on the device, and the user (witness) checks it.

[0863] Input: Generated caricature data

[0864] How it works: The device displays a sketch to a witness, who reviews it and tells the police officer what corrections they would like to make.

[0865] Output: The witness' desired correction score is obtained.

[0866] Step 5:

[0867] The user (police officer) inputs the desired corrections (e.g., "the eyes should be narrower") into the device's UI.

[0868] Input: Witness' desired correction point

[0869] Operation: The device sends the desired corrections to the server.

[0870] Output: The desired correction points are sent to the server for processing.

[0871] Step 6:

[0872] The server updates the portrait based on the desired corrections.

[0873] Input: desired corrections

[0874] How it works: The generative AI model on the server modifies the portrait, taking into account the desired corrections.

[0875] Output: The corrected portrait data is created and sent to the device.

[0876] Step 7:

[0877] The updated portrait is displayed on the device, and the user (witness) checks it again.

[0878] Input: Corrected portrait data

[0879] How it works: The device displays the revised portrait to the witness. If the witness is not satisfied, the process returns to input further revisions.

[0880] Output: The witness's desired correction points are obtained, or the final portrait is confirmed by the witness.

[0881] Step 8:

[0882] The user (police officer) checks the final portrait and selects "Save" if they are satisfied.

[0883] Input: Final caricature approval

[0884] Operation: The device sends a "save" request to the server, which saves the final portrait data to a database.

[0885] Output: A notification that the final portrait has been saved will be displayed on your device.

[0886] By following these processing steps, a system is realized that can quickly and accurately generate and modify caricatures in real time.

[0887] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0888] This invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on characteristic information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[0889] This system includes a series of processes for acquiring feature information from witnesses and generating and modifying caricatures, as well as a function for recognizing the emotions of witnesses and dynamically adjusting the interface and generation process accordingly.The system configuration and its specific program processing are explained below.

[0890] System Configuration

[0891] The system consists of four main components:

[0892] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[0893] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[0894] 3. Terminal: An input device used by witnesses and police officers.

[0895] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[0896] Program processing overview

[0897] 1. Initial Setup

[0898] The terminal starts up and the officer enters their login information.

[0899] The server verifies the login information and authentication is complete.

[0900] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0901] A message will appear on the device indicating that the initial setup is complete.

[0902] 2. Hearing from witnesses

[0903] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0904] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[0905] The terminal transmits input data to the server in real time.

[0906] 3. Eyewitness emotion recognition

[0907] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[0908] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[0909] 4. Caricature Generation

[0910] The server generates an initial portrait based on the received feature information.

[0911] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[0912] 5. Adjustments and Feedback

[0913] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0914] The user (police officer) inputs the desired corrections into the device's UI.

[0915] The terminal transmits the correction data to the server.

[0916] The server updates the portrait based on the correction data and sends it back to the device.

[0917] The corrected portrait will be displayed on the device, and this process will be repeated until the witness is satisfied.

[0918] 6. Final confirmation and saving

[0919] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[0920] The user (police officer) selects "Save."

[0921] The server stores the final portrait data in a database.

[0922] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[0923] Specific examples

[0924] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[0925] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[0926] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[0927] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[0928] 4. The server generates the first portrait based on the feature information and sends it to the device.

[0929] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[0930] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[0931] 7. The server updates the portrait based on the correction data and sends it back to the device.

[0932] 8. Repeat this process until you have created a portrait that satisfies the witness.

[0933] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[0934] The system allows witnesses to provide feedback in a relaxed manner, improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[0935] The processing flow will be explained below.

[0936] Step 1:

[0937] The terminal starts up and the officer enters their login information.

[0938] Step 2:

[0939] The server verifies the login information and authentication is complete.

[0940] Step 3:

[0941] The server establishes a connection to the database and initializes the caricature generation algorithm.

[0942] Step 4:

[0943] A message will appear on the device indicating that the initial setup is complete.

[0944] Step 5:

[0945] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[0946] Step 6:

[0947] The user (police officer) inputs the obtained characteristic information into the terminal's UI.

[0948] Step 7:

[0949] The terminal transmits input data to the server in real time.

[0950] Step 8:

[0951] The server generates an initial portrait based on the received feature information.

[0952] Step 9:

[0953] The server transmits the generated portrait data to the terminal.

[0954] Step 10:

[0955] The first portrait will be displayed on the device.

[0956] Step 11:

[0957] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[0958] Step 12:

[0959] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[0960] Step 13:

[0961] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[0962] Step 14:

[0963] The user (police officer) inputs the desired corrections into the device's UI.

[0964] Step 15:

[0965] The terminal transmits the correction data to the server.

[0966] Step 16:

[0967] The server updates the portrait based on the correction data.

[0968] Step 17:

[0969] The server transmits the updated portrait data to the terminal again.

[0970] Step 18:

[0971] The revised portrait will be displayed on the device.

[0972] Step 19:

[0973] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[0974] Step 20:

[0975] The users (witnesses and police officers) agree on the final sketch.

[0976] Step 21:

[0977] The user (police officer) selects "Save."

[0978] Step 22:

[0979] The server stores the final portrait data in a database.

[0980] Step 23:

[0981] The server notifies the terminal that the save is complete.

[0982] Step 24:

[0983] A message will appear on your device indicating that the save is complete.

[0984] Example 2

[0985] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0986] In conventional portrait generation systems, the input of information from witnesses and the correction of portraits are often cumbersome and time-consuming. Furthermore, witness tension and anxiety can affect the accuracy of the portrait. For this reason, there is a need to create an environment where witnesses can relax and provide detailed feedback. Furthermore, because it is difficult to generate and correct portraits in real time, there is a need for highly accurate and rapid investigative support.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0988] In this invention, the server includes means for recognizing emotions from the facial expressions and voices of witnesses, means for dynamically adjusting a user interface based on the recognized emotion information, means for generating a caricature based on input feature information, and means for modifying the caricature based on input feedback. This allows the witness to provide feedback in a relaxed state, and enables the generation and modification of highly accurate caricatures in real time.

[0989] "Means for inputting characteristic information from witnesses" refers to an interface or device that allows witnesses to provide information about the appearance and characteristics of the perpetrator to the system.

[0990] "Means for generating a caricature based on input feature information" refers to an algorithm or program for automatically creating a caricature based on appearance and features input by a witness.

[0991] "Means for displaying the generated caricature to a witness" refers to an interface or device for showing the generated caricature to a witness through a display or monitor.

[0992] "Means for inputting feedback from witnesses" refers to an interface or device that allows witnesses to input their opinions and comments to the system for corrections and improvements to the generated portrait.

[0993] "Means for modifying the portrait based on input feedback" refers to an algorithm or program that automatically modifies the portrait based on feedback provided by witnesses.

[0994] "Means for displaying the modified portrait to the witness again" refers to an interface or device for displaying the modified portrait to the witness again through a display or monitor.

[0995] "Means for storing the final portrait in a database" refers to a system or program for storing the finalized portrait in a database in digital format.

[0996] "Means for recognizing emotions from the facial expressions and voice of witnesses" refers to emotion recognition algorithms or programs that analyze the facial expressions and voice of witnesses to identify their emotional state (e.g., tension, anxiety, relaxation) in real time.

[0997] "Means for dynamically adjusting a user interface based on recognized emotional information" refers to a system or program for automatically modifying or adjusting an interface in response to a recognized emotional state of a witness to allow the user to relax and provide feedback.

[0998] "Server" refers to a computer system that is a central processing unit for executing each of the above means and is responsible for major processes such as receiving, transmitting, and storing data, as well as generating and modifying portraits.

[0999] "Device" refers to the hardware device used by witnesses and police officers to input characteristic information and feedback and to display the sketch, including, for example, a computer, tablet, or smartphone.

[1000] An "emotion engine" refers to software or algorithms that analyze facial expressions and voice of witnesses to recognize their emotions and dynamically adjust the user interface or system behavior based on those emotions.

[1001] "User Interface (UI)" refers to the screen displays and controls used by witnesses and police officers to interact with the system, including the design and functionality to facilitate inputting information and providing feedback.

[1002] The present invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on feature information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[1003] System Configuration

[1004] The system consists of four main components:

[1005] 1. User Interface (UI): The screen through which witnesses and investigators interact.

[1006] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1007] 3. Terminal: An input device used by witnesses and police officers.

[1008] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[1009] Program processing

[1010] The server includes a means for inputting feature information from a witness, a means for generating a caricature based on the input feature information, a means for displaying the generated caricature to the witness, a means for inputting feedback from the witness, a means for modifying the caricature based on the feedback, a means for displaying the modified caricature again to the witness, and a means for saving the final caricature in a database. The system also includes an emotion engine for recognizing emotions from the facial expressions and voices of the witness, and a means for dynamically adjusting the UI based on the recognition results.

[1011] Hardware and Software

[1012] Hardware: The system components include a PC, a tablet, and a networked server.

[1013] Software: Includes emotion recognition algorithms, similar face generation algorithms, and database management systems.

[1014] Specific examples

[1015] As an example of implementation, we will explain a case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." Here, we will specifically explain the input process, emotion recognition process, caricature generation process, and correction process.

[1016] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1017] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[1018] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[1019] 4. The server generates the first portrait based on the feature information and sends it to the device.

[1020] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1021] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[1022] 7. The server updates the portrait based on the correction data and sends it back to the device.

[1023] 8. Repeat this process until you have created a portrait that satisfies the witness.

[1024] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1025] Prompt Sentence Examples

[1026] Below are some examples of specific prompts that can be input to the generative AI model in this system:

[1027] "A witness listed the suspect's characteristics as 'black hair, a round face, and narrow eyes.' Please generate an initial sketch based on this information."

[1028] "We have detected the witness' emotional state. They do not appear to be relaxed. Please adjust the user interface to help them relax."

[1029] "A witness pointed out a correction to the portrait, saying 'the eyes are narrower.' Please generate a new portrait that reflects this."

[1030] This invention allows witnesses to provide feedback in a relaxed state, improving the speed and accuracy of investigations and reducing the risk of false accusations.

[1031] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1032] Program processing flow

[1033] Step 1: Initial Setup

[1034] Step 1-1: Start the terminal and log in

[1035] Input: Police officer's login information (ID and password)

[1036] Output: Server authentication result

[1037] Specific operation:

[1038] The device will start up and display the login screen.

[1039] The user (police officer) enters their ID and password and clicks the "Login" button.

[1040] The terminal sends the input information to the server, which verifies the login information.

[1041] The server sends the authentication result to the terminal, and the terminal displays a message indicating "authentication successful" or "authentication failed."

[1042] Step 1-2: Establishing a database connection

[1043] Input: Server authentication success result

[1044] Output: Database connection established message

[1045] Specific operation:

[1046] Once the server receives the successful authentication message, it attempts to connect to the database.

[1047] The server initializes the caricature generation algorithm.

[1048] The server sends a message to the terminal indicating that the database connection has been successful, and the terminal displays the message "Initial settings completed."

[1049] Step 2: Hear from witnesses

[1050] Step 2-1: Interviewing witnesses

[1051] Input: Basic information of the witness (e.g., age, gender, hair color, etc.)

[1052] Output: Input data of eyewitness characteristics

[1053] Specific operation:

[1054] The user (police officer) asks witnesses questions about the basic characteristics of the perpetrator.

[1055] The user (police officer) enters the witness's answers into the device's UI.

[1056] Step 2-2: Real-time transmission of feature information

[1057] Input: Witness characteristics information entered into the terminal

[1058] Output: Feature information data sent to the server

[1059] Specific operation:

[1060] The device transmits the entered characteristic information of the witness to the server in real time.

[1061] The server receives the characteristic information and temporarily stores it in a database.

[1062] Step 3: Eyewitness emotion recognition

[1063] Step 3-1: Acquiring emotion data

[1064] Input: witness' facial expressions and voice data

[1065] Output: Recognized emotion information

[1066] Specific operation:

[1067] The emotion engine scans the facial expressions and voices of witnesses in real time via the device.

[1068] The emotion engine analyzes the collected data to recognize the witness's emotions (e.g., nervous, anxious, relaxed).

[1069] The emotion engine sends the recognition results to the server.

[1070] Step 3-2: Dynamic UI adjustment

[1071] Input: Recognized emotion information

[1072] Output: Adjusted user interface

[1073] Specific operation:

[1074] The server analyzes emotional information and determines UI settings that are easy to relax.

[1075] The server sends instructions to the terminal to change the UI settings, and the terminal dynamically adjusts the UI.

[1076] Step 4: Generate a caricature

[1077] Step 4-1: Generate the first portrait

[1078] Input: Witness characteristics information

[1079] Output: First portrait data

[1080] Specific operation:

[1081] The server runs an algorithm to generate an initial portrait based on the feature information.

[1082] The server transmits the generated portrait data to the terminal.

[1083] Step 4-2: Displaying the caricature

[1084] Input: First portrait data

[1085] Output: Initial portrait shown to witnesses

[1086] Specific operation:

[1087] The first portrait received by the device is displayed on the screen.

[1088] Step 5: Adjustments and feedback

[1089] Step 5-1: Input desired corrections

[1090] Input: Eyewitness feedback

[1091] Output: Data of desired correction points

[1092] Specific operation:

[1093] The user (witness) checks the sketch and tells the user (police officer) what needs to be corrected.

[1094] The user (police officer) inputs the desired corrections into the UI.

[1095] Step 5-2: Submitting the correction data

[1096] Input: Correction requests entered into the UI

[1097] Output: Corrected data sent to the server

[1098] Specific operation:

[1099] The terminal transmits the desired corrections to the server.

[1100] Step 5-3: Update your portrait

[1101] Input: Data of desired correction points

[1102] Output: Updated caricature data

[1103] Specific operation:

[1104] The server runs an algorithm to update the portrait based on the correction data.

[1105] The server transmits the updated portrait data to the terminal again.

[1106] Step 5-4: Displaying the updated caricature

[1107] Input: Updated caricature data

[1108] Output: Updated portrait displayed to witnesses

[1109] Specific operation:

[1110] The updated portrait received by the terminal is displayed on the display.

[1111] This process is repeated until the witness is satisfied.

[1112] Step 6: Final review and save

[1113] Step 6-1: Final confirmation

[1114] Input: Eyewitness confirmation of updated portrait

[1115] Output: Final caricature data

[1116] Specific operation:

[1117] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[1118] Step 6-2: Save the portrait

[1119] Input: Final portrait data

[1120] Output: Caricature data stored in a database

[1121] Specific operation:

[1122] The user (police officer) selects "Save."

[1123] The server stores the final portrait data in a database.

[1124] Step 6-3: Notification of save completion

[1125] Input: Database save completion result

[1126] Output: Message notifying you that saving is complete

[1127] Specific operation:

[1128] The server notifies the terminal that the save is complete, and the message "Saving complete" is displayed on the terminal.

[1129] (Application example 2)

[1130] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1131] In criminal investigations, conventional methods of generating sketches based on eyewitness testimony can lead to inaccurate testimony due to the eyewitness's emotions or nervousness, and can hinder smooth communication with investigators. Furthermore, generating and modifying sketches in real time is difficult, resulting in a lack of speed and accuracy in investigations. To solve these issues, a system is needed that can recognize eyewitness emotions and dynamically adjust the interface to generate and modify sketches.

[1132] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1133] In this invention, the server includes means for recognizing emotions from the facial expressions and voice of a witness, means for dynamically adjusting the interface based on the recognized emotion information, and means for generating a portrait based on the input feature information. This allows the witness to give accurate testimony in a relaxed state, and enables rapid and highly accurate generation of a portrait.

[1134] A "witness" is someone who directly witnessed an incident or accident, or was involved in it face-to-face.

[1135] "Characteristic information" refers to information that records details of the appearance and behavior of a person described by a witness.

[1136] A "caricature" is an illustration or image of a person's face based on eyewitness testimony.

[1137] "Facial expression" refers to temporary emotions and reactions expressed through facial muscle movements, gaze, etc.

[1138] "Voice" refers to the linguistic expressions such as tone, intonation, and volume of the voice spoken by the witness.

[1139] "Emotional information" refers to data that represents the psychological state of a witness analyzed by the emotion engine.

[1140] "Interface" refers to the screen and operating means for exchanging information between the user and the system.

[1141] "Feedback" refers to the behavior of witnesses expressing their opinions or requests for corrections regarding the generated portrait.

[1142] "Server" refers to a remote computer for storing, processing, and transmitting data.

[1143] This invention is a system that acquires feature information based on eyewitness testimony and uses an emotion engine to generate and modify highly accurate caricatures in real time. The system takes into account the emotions of the eyewitness and modifies the caricature as many times as necessary until the eyewitness is satisfied, and finally saves it in a database.

[1144] System programs and the hardware and software used

[1145] The system consists of the following main components:

[1146] 1. Server: Generates and modifies caricatures based on feature and emotion information.

[1147] Software used: Emotion recognition software, caricature generation algorithm

[1148] Hardware used: high-speed processor, database server

[1149] 2. Terminal: An input device used by witnesses or investigators to transmit input feature information and feedback information to the server. Examples include smart glasses, smartphones, and tablets.

[1150] Software used: Input interface, emotion recognition software

[1151] Hardware used: Smart glasses (e.g., Google Glass, Vuzix Blade), smartphone (iOS / Android)

[1152] 3. User Interface (UI): A screen for witness and investigator interaction, displaying the sketch generated in real time and providing the ability to input witness feedback.

[1153] Software used: Browser-based UI, desktop application, mobile app

[1154] Explanation of program processing

[1155] The server uses emotion recognition software to recognize emotions from the witness's facial expressions and voice. This software analyzes the witness's facial expression data and voice data to recognize their current emotional state (e.g., nervous, anxious, relaxed). Based on the emotional information, the server dynamically adjusts the interface to provide an environment in which the witness can relax and easily testify.

[1156] When feature information is entered on the device, it is sent to the server, which then generates an initial caricature based on the feature information using a caricature generation algorithm. The generated caricature is then displayed to the witness through a user interface. When the witness provides feedback, the device sends the information to the server, which then modifies the caricature based on the feedback.

[1157] This process is repeated until the witness is satisfied, and once the final portrait is completed, the server stores it in a database.

[1158] Specific examples

[1159] For example, if a witness says, "The perpetrator has black hair, a round face, and narrow eyes," these characteristics are entered into the device. The server uses emotion recognition software to analyze whether the witness is nervous or relaxed. If it detects nervousness, the server adjusts the user interface, changing the interface design and the questions presented to put the witness at ease.

[1160] After the first sketch is generated and displayed on the device, the witness may point out that the eyes are narrower. This feedback is sent to the server, which then modifies the sketch based on the feedback. This process is repeated several times until a final sketch is created that satisfies the witness.

[1161] Prompt Sentence Examples

[1162] "Please tell us the characteristics of the perpetrator. Please be as detailed as possible. What is the shape of the eyes? What color is the hair? What is the shape of the face? Please relax and answer."

[1163] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1164] Step 1:

[1165] The server initializes the system. The terminal starts up and the investigator enters their login information. Once the login information is entered, the server verifies it and performs authentication. Once authentication is complete, the server establishes a connection to the database and initializes the portrait generation algorithm. At this point, the terminal displays a message indicating that initialization is complete.

[1166] Input: Investigator login information

[1167] Output: Authentication complete message, initial setup complete message

[1168] Data processing / calculation: Authenticating login information, establishing database connections, initializing algorithms

[1169] Step 2:

[1170] The user (investigator) asks the witness for basic information (e.g., age, gender, hair color, etc.) in an interview format. The basic information provided by the witness is entered into the device's UI. The device then transmits the entered characteristic information to the server in real time.

[1171] Input: basic information and characteristics of the witness

[1172] Output: Sending the input feature information

[1173] Data processing / calculation: Input and organization of basic information, real-time transmission of characteristic information

[1174] Step 3:

[1175] The server receives the feature information and uses an emotion engine to recognize emotions from the witness's facial expressions and voice. The recognized emotion information is analyzed on the server side, and the UI is dynamically adjusted to make the witness feel relaxed and easy to give feedback.

[1176] Input: witness's facial expression data, voice data

[1177] Output: Recognized emotion information

[1178] Data processing / calculation: Analysis of facial expression data and voice data, emotion recognition, dynamic UI adjustment

[1179] Step 4:

[1180] The server generates an initial portrait based on the received feature information. A portrait generation algorithm is used for this generation. The generated portrait data is sent to the terminal, and the initial portrait is displayed on the terminal.

[1181] Input: feature information

[1182] Output: Sending and displaying the initial portrait data

[1183] Data processing / calculation: Generate portraits based on feature information, send portrait data

[1184] Step 5:

[1185] The user (witness) checks the sketch and tells the investigator what they would like to be corrected (e.g., "Make the eyes thinner"). The investigator inputs the witness's feedback into the device's UI. The input correction requests are sent from the device to the server.

[1186] Input: Witness feedback, correction requests

[1187] Output: Sending correction requests

[1188] Data processing / calculation: Input and sorting of feedback, sending correction requests

[1189] Step 6:

[1190] The server then modifies the portrait based on the requested corrections. The modified portrait data is then sent back to the terminal and displayed on the device. This process is repeated until the witness is satisfied.

[1191] Input: desired corrections

[1192] Output: Send and display the corrected portrait data

[1193] Data processing / calculation: Correction of portraits based on desired correction points, resending of portrait data

[1194] Step 7:

[1195] The users (witness and investigator) review the final sketch and, if satisfied, agree on it as the final sketch. The investigator selects "Save," and the server saves the final sketch data in the database. A notification that saving is complete is displayed on the device.

[1196] Input: Final portrait data

[1197] Output: Database save, save completion notification

[1198] Data processing / calculation: Saving portrait data and sending a notification when saving is complete

[1199] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1200] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1201] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1202] [Third embodiment]

[1203] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1204] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1205] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1206] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1207] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1209] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1210] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1211] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1212] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1213] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1214] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1215] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[1216] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[1217] System Configuration

[1218] The system consists of three main components:

[1219] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[1220] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1221] 3. Terminal: An input device used by witnesses and police officers.

[1222] Program processing overview

[1223] 1. Initial Setup

[1224] The terminal starts up and the officer logs in.

[1225] The server verifies the login information and authentication is complete.

[1226] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1227] A message will appear on the device indicating that the initial setup is complete.

[1228] 2. Hearing from witnesses

[1229] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[1230] The terminal transmits input data to the server in real time.

[1231] 3. Caricature Generation

[1232] The server generates an initial portrait based on the received feature information.

[1233] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[1234] 4. Adjustments and feedback

[1235] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1236] The user (police officer) inputs the desired corrections into the device's UI.

[1237] The terminal transmits the correction data to the server.

[1238] The server updates the portrait based on the correction data, and transmits the updated portrait data to the terminal again.

[1239] The updated portrait will be displayed on the device, and the process will be repeated until the witness is satisfied.

[1240] 5. Final confirmation and saving

[1241] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[1242] The user (police officer) selects "Save."

[1243] The server stores the final portrait data in a database.

[1244] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[1245] Specific examples

[1246] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[1247] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1248] 2. The server generates the first portrait based on the feature information and sends it to the device.

[1249] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1250] 4. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[1251] 5. The server updates the portrait based on the correction data and sends it back to the device.

[1252] 6. Repeat this process until you have created a portrait that satisfies the witness.

[1253] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1254] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[1255] The processing flow will be explained below.

[1256] Step 1:

[1257] The terminal starts up and the officer enters their login information.

[1258] Step 2:

[1259] The server verifies the login information and authentication is complete.

[1260] Step 3:

[1261] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1262] Step 4:

[1263] A message will appear on the device indicating that the initial setup is complete.

[1264] Step 5:

[1265] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[1266] Step 6:

[1267] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[1268] Step 7:

[1269] The terminal transmits the input characteristic information to the server in real time.

[1270] Step 8:

[1271] The server generates an initial portrait based on the received feature information.

[1272] Step 9:

[1273] The server transmits the generated portrait data to the terminal.

[1274] Step 10:

[1275] The first portrait will be displayed on the device.

[1276] Step 11:

[1277] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1278] Step 12:

[1279] The user (police officer) inputs the desired corrections into the device's UI.

[1280] Step 13:

[1281] The terminal transmits the correction data to the server.

[1282] Step 14:

[1283] The server updates the portrait based on the correction data.

[1284] Step 15:

[1285] The server transmits the updated portrait data to the terminal again.

[1286] Step 16:

[1287] The revised portrait will be displayed on the device.

[1288] Step 17:

[1289] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[1290] Step 18:

[1291] The users (witnesses and police officers) agree on the final sketch.

[1292] Step 19:

[1293] The user (police officer) selects "Save."

[1294] Step 20:

[1295] The server stores the final portrait data in a database.

[1296] Step 21:

[1297] The server notifies the terminal that the save is complete.

[1298] Step 22:

[1299] A message will appear on your device indicating that the save is complete.

[1300] Example 1

[1301] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1302] In criminal investigations, the task of creating a portrait based on eyewitness testimony is time-consuming and labor-intensive. Furthermore, eyewitnesses' memories are often vague, requiring repeated revisions, making it difficult to quickly generate a highly accurate portrait. To address these challenges, a technology is needed that can generate and revise portraits in real time based on information obtained from eyewitnesses, ultimately resulting in a highly accurate portrait.

[1303] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1304] In this invention, the server includes a means for a user to input feature information from a witness, a means for generating a portrait using a generative AI model based on the input feature information, and a means for displaying the corrected portrait to the user again. This allows for the generation and correction of portraits based on witness testimony in real time, thereby improving the efficiency and accuracy of investigations.

[1305] The "user" is the person who operates the system, inputs information from witnesses, and reviews the generated and modified portraits.

[1306] A "witness" is someone who saw a suspect at a crime scene or elsewhere, and whose testimony is used to generate or modify the sketch.

[1307] "Characteristic information" refers to specific physical characteristics of a suspect as testified by witnesses (e.g., age, gender, hair color, face shape, eye shape, etc.).

[1308] The "server" is the central computing device of the system, a combination of hardware and software that receives feature information, generates and modifies portraits using generative AI models, and stores data.

[1309] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate and modify caricatures in real time using feature information as input.

[1310] A "caricature" is an image of a suspect's face generated from witness testimony and can be digitally displayed and modified.

[1311] "Feedback" is information including desired corrections and suggestions made by the witness to the generated portrait.

[1312] A "database" is a digital storage system that organizes and stores similar or related data and allows it to be retrieved as needed.

[1313] "Real-time" refers to a state in which data is generated or modified immediately and the results are reflected immediately to the user.

[1314] "Input means" refers to an interface or device that allows a user to input feature information and feedback to the system.

[1315] The "display means" refers to a screen or monitor that visually presents the generated portrait and the correction results to the user.

[1316] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[1317] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[1318] System Configuration

[1319] The system consists of three main components:

[1320] 1. User Interface (UI): The screen through which witnesses and users interact.

[1321] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1322] 3. Terminal: An input device used by witnesses and users.

[1323] The system operates as follows.

[1324] 1. Initial Setup

[1325] The terminal starts up and the user (police officer) logs in. The server verifies the login information and authentication is completed. The server establishes a connection to the database and initializes the caricature generation algorithm. A message indicating initial setup is complete is displayed on the terminal.

[1326] 2. Hearing from witnesses

[1327] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the information into the device's UI. The device then sends the input data to the server in real time.

[1328] 3. Caricature Generation

[1329] The server generates an initial portrait based on the received feature information, and then transmits the generated portrait data to the terminal, where the initial portrait is displayed.

[1330] 4. Adjustments and feedback

[1331] The user (witness) checks the sketch and tells the user (police officer) what corrections they would like to make (e.g., "Make the eyes thinner"). The user (police officer) enters the corrections they would like to make into the device's UI. The device sends the correction data to the server. The server updates the sketch based on the correction data, and the server sends the updated sketch data back to the device. The updated sketch is displayed on the device. This process is repeated until the witness is satisfied.

[1332] 5. Final confirmation and saving

[1333] The users (witness and police officer) review the final sketch, and if they are satisfied, they agree on it as the final sketch. The user (police officer) selects "Save." The server saves the final sketch data in the database. The server notifies the device that saving is complete, and a message indicating this is displayed on the device.

[1334] Specific examples

[1335] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[1336] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1337] 2. The server generates the first portrait based on the feature information and sends it to the device.

[1338] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1339] 4. The user (police officer) inputs the correction, such as "the eyes should be narrower," and the device sends the correction information to the server.

[1340] 5. The server updates the portrait based on the correction data and sends it back to the device.

[1341] 6. Repeat this process until you have created a portrait that satisfies the witness.

[1342] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1343] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[1344] Prompt Sentence Examples

[1345] markdown

[1346] This AI model is used to generate a portrait based on witness testimony, which includes the following:

[1347] Hair color: Black

[1348] Face Shape: Round

[1349] Eye shape: Narrow

[1350] Use these features to generate an initial caricature, then follow the instructions to make any necessary corrections.

[1351] In this way, this system uses artificial intelligence technology to quickly and accurately convert eyewitness testimony into a facial sketch, dramatically improving the efficiency of criminal investigations.

[1352] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1353] Step 1: Initial Setup

[1354] The terminal boots up and the user accesses a login screen, which takes a username and password as input.

[1355] The server receives the username and password and verifies the authentication information using an SQL database. The authentication information is verified as part of the data processing, and the authentication result (success / failure) is obtained as the output.

[1356] If the server is successful in authentication, it establishes a connection to the database and initializes the caricature generation algorithm. The output is the connection status and the initialization completion status of the algorithm.

[1357] A message indicating that initial setup is complete is displayed on the terminal, and the user confirms that the system is ready.

[1358] Step 2: Interview witnesses

[1359] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and obtains characteristic information from the witness as input.

[1360] The user (police officer) inputs the feature information into the device's UI. The feature information is received as input data.

[1361] The terminal transmits the input characteristic information to the server in real time. The characteristic information is received as data processing, and the characteristic information is output and transmitted to the server.

[1362] Step 3: Generate a caricature

[1363] The server receives the feature information sent from the device and inputs it into the generative AI model, using the feature information as input data.

[1364] The server uses the generative AI model to generate an initial portrait based on the feature information. The generation process is carried out as a data calculation, and the generated portrait data is obtained as the output.

[1365] The server transmits the generated portrait data to the terminal.

[1366] The device displays the sketch on the screen, and the user (police officer) asks a witness to confirm the sketch.

[1367] Step 4: Adjustments and feedback

[1368] The user (witness) checks the generated sketch and tells the police officer what they would like to improve (e.g., "Make the eyes thinner"). Feedback information is received as input.

[1369] The user (police officer) inputs the desired corrections into the device's UI. The correction information is received as input data.

[1370] The terminal transmits the correction data to the server. The correction information is received as data processing, and the correction data is output and transmitted to the server.

[1371] The server re-executes the generative AI model based on the received correction data, and updates the portrait. The update process is performed as data calculation, and the updated portrait data is obtained as output.

[1372] The server transmits the updated portrait data to the terminal.

[1373] The device displays the updated sketch on the screen, and the user (police officer) asks the witness to review the updated sketch. This process is repeated until the witness is satisfied.

[1374] Step 5: Final review and save

[1375] The users (witnesses and police officers) review the final sketch and, if they are satisfied, agree on it as the final version.,Final confirmation information is received as input.

[1376] The user (police officer) selects "Save." A save instruction is received as an operational input.

[1377] The server saves the final portrait data in the database. The data saving process is performed as data processing, and the saving completion status is obtained as output.

[1378] The server notifies the terminal that the save is complete.

[1379] The terminal displays a message indicating that the data has been saved, and the process ends. The user (police officer) confirms that the process has been completed successfully.

[1380] In this way, through the specific actions and data processing performed at each step, the system can generate and modify highly accurate portraits in real time based on eyewitness testimony.

[1381] (Application example 1)

[1382] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1383] Traditional criminal investigation sketch creation processes are often manual, making it difficult to quickly and accurately generate sketches. Furthermore, due to a lack of a mechanism for incorporating real-time feedback from witnesses, it is difficult to quickly create a final sketch that satisfies the witnesses. This makes it difficult to effectively respond in situations where speed and accuracy are required in investigations.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1385] In this invention, the server includes means for inputting feature information from a witness, means for generating a portrait based on the input feature information, means for displaying the generated portrait to the witness, means for inputting feedback from the witness, means for modifying the portrait based on the input feedback, means for displaying the modified portrait again to the witness, means for saving the final portrait in a database, means for transmitting and receiving feature information to the server in real time using a smart device, means for transmitting and receiving feedback to the server in real time, means for inputting feature information and feedback using prompt sentences, and means for generating and modifying a portrait based on the feature information and feedback using a generative AI model, thereby enabling rapid and highly accurate generation and modification of portraits.

[1386] "Characteristic information" is information provided by witnesses that indicates the criminal's external features and characteristics.

[1387] A "portrait" is a drawing that represents an image of the criminal's face, generated based on feature information.

[1388] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate and modify caricatures based on feature information and feedback.

[1389] A "prompt" is a phrase that guides witnesses and police officers when entering characteristic information and feedback.

[1390] "Feedback" is information indicating corrections or requests for improvement provided by a witness to the displayed portrait.

[1391] "Smart devices" refer to portable electronic devices with advanced functions, such as smartphones and tablets.

[1392] "Server" refers to a central processing unit that receives feature information and feedback, generates and modifies caricatures, and stores them in a database.

[1393] An "eyewitness" is someone who actually witnesses a crime scene or the circumstances of an incident and provides testimony.

[1394] The "database" is a data storage system for permanently storing the generated portraits and related information.

[1395] An "input device" is a hardware device used by a witness or police officer to input characteristic information or feedback.

[1396] This invention is a system that quickly generates a portrait based on characteristic information obtained from witnesses and makes corrections as necessary. This system generates a portrait in real time based on witness testimony, supporting investigations with high accuracy and efficiency.

[1397] System configuration

[1398] The system of the present invention comprises the following main components:

[1399] 1. User Interface (UI): A screen for interaction between witnesses and police officers. Implemented on smart devices (smartphones and tablets), it includes a testimony input form, a portrait display area, adjustment buttons, etc.

[1400] 2. Server: A central processing unit that processes caricature generation. It receives feature information and feedback, and generates and modifies caricatures using generative AI models.

[1401] 3. Database: A storage system for storing the generated portraits and related information.

[1402] Program processing overview

[1403] Hardware and software used

[1404] Hardware: Smart devices (smartphones, tablets), servers

[1405] Software: React Native or Swift UI (user interface), Flask (backend), TensorFlow (generative AI models), MySQL or MongoDB (database)

[1406] Data processing and calculation

[1407] Input and transmission of characteristic information: The user inputs the characteristic information of the witness through the UI on the smart device, and this information is transmitted to the server in real time.

[1408] Caricature generation: The server generates a caricature using TensorFlow based on the received feature information. The generated caricature is sent back to the smart device and displayed on the UI.

[1409] Feedback acquisition and revision: The witness enters feedback (requests for revision) on the portrait, which is then sent back to the server. The server then revises the portrait based on the feedback and sends the updated portrait back to the smart device. This process is repeated until the witness is satisfied.

[1410] Final saving: Once a final satisfactory caricature is generated, it is saved in the database.

[1411] Specific examples

[1412] For example, consider the case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." The user inputs characteristics such as "black hair," "round face," and "narrow eyes" into the UI of their smart device, which is then sent to the server. The server generates an initial sketch based on this characteristic information and sends it back to the smart device. If the witness provides feedback such as "his eyes are a little narrower," the sketch is sent again to the server, and the sketch is revised and reviewed by the witness. This process is repeated until the witness is satisfied, and a satisfactory sketch is generated and saved.

[1413] Prompt Sentence Examples

[1414] Text prompt:

[1415] TXT

[1416] Testimony input prompt:

[1417] Please enter the characteristics information provided by the witness in the following format:

[1418] Name: [Witness A]

[1419] Age: [Around 30 years old]

[1420] Hair color: [Black]

[1421] Eye shape: [narrow]

[1422] Once you have completed the input, please press the "Submit" button.

[1423] Caricature correction prompt:

[1424] Please review your current portrait and enter any necessary corrections in the following format:

[1425] Correction: [Eyes are a little narrower]

[1426] Reason for revision: [To make the impression of the actual perpetrator more realistic]

[1427] Once you have completed the entry, press the "Edit" button.

[1428] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1429] Step 1:

[1430] The terminal starts up and the officer logs in.

[1431] Input: Police officer's login information (username, password)

[1432] Operation: The device sends login information to the server. The server verifies the login information and performs authentication. If authentication is successful, the device displays a message indicating initial setup is complete.

[1433] Output: A notification that initial setup is complete will be displayed on the device.

[1434] Step 2:

[1435] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[1436] Input: Witness characteristics (e.g., age, gender, hair color, face shape, eye shape)

[1437] Operation: The device sends the input feature information to the server in real time.

[1438] Output: The feature information is sent to the server for processing.

[1439] Step 3:

[1440] The server generates an initial portrait based on the received feature information.

[1441] Input: feature information

[1442] How it works: The generative AI model on the server generates a portrait based on feature information.

[1443] Output: The generated portrait data is created and sent to the device.

[1444] Step 4:

[1445] The initial portrait is displayed on the device, and the user (witness) checks it.

[1446] Input: Generated caricature data

[1447] How it works: The device displays a sketch to a witness, who reviews it and tells the police officer what corrections they would like to make.

[1448] Output: The witness' desired correction score is obtained.

[1449] Step 5:

[1450] The user (police officer) inputs the desired corrections (e.g., "the eyes should be narrower") into the device's UI.

[1451] Input: Witness' desired correction point

[1452] Operation: The device sends the desired corrections to the server.

[1453] Output: The desired correction points are sent to the server for processing.

[1454] Step 6:

[1455] The server updates the portrait based on the desired corrections.

[1456] Input: desired corrections

[1457] How it works: The generative AI model on the server modifies the portrait, taking into account the desired corrections.

[1458] Output: The corrected portrait data is created and sent to the device.

[1459] Step 7:

[1460] The updated portrait is displayed on the device, and the user (witness) checks it again.

[1461] Input: Corrected portrait data

[1462] How it works: The device displays the revised portrait to the witness. If the witness is not satisfied, the process returns to input further revisions.

[1463] Output: The witness's desired correction points are obtained, or the final portrait is confirmed by the witness.

[1464] Step 8:

[1465] The user (police officer) checks the final portrait and selects "Save" if they are satisfied.

[1466] Input: Final caricature approval

[1467] Operation: The device sends a "save" request to the server, which saves the final portrait data to a database.

[1468] Output: A notification that the final portrait has been saved will be displayed on your device.

[1469] By following these processing steps, a system is realized that can quickly and accurately generate and modify caricatures in real time.

[1470] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1471] This invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on characteristic information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[1472] This system includes a series of processes for acquiring feature information from witnesses and generating and modifying caricatures, as well as a function for recognizing the emotions of witnesses and dynamically adjusting the interface and generation process accordingly.The system configuration and its specific program processing are explained below.

[1473] System Configuration

[1474] The system consists of four main components:

[1475] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[1476] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1477] 3. Terminal: An input device used by witnesses and police officers.

[1478] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[1479] Program processing overview

[1480] 1. Initial Setup

[1481] The terminal starts up and the officer enters their login information.

[1482] The server verifies the login information and authentication is complete.

[1483] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1484] A message will appear on the device indicating that the initial setup is complete.

[1485] 2. Hearing from witnesses

[1486] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[1487] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[1488] The terminal transmits input data to the server in real time.

[1489] 3. Eyewitness emotion recognition

[1490] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[1491] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[1492] 4. Caricature Generation

[1493] The server generates an initial portrait based on the received feature information.

[1494] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[1495] 5. Adjustments and Feedback

[1496] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1497] The user (police officer) inputs the desired corrections into the device's UI.

[1498] The terminal transmits the correction data to the server.

[1499] The server updates the portrait based on the correction data and sends it back to the device.

[1500] The corrected portrait will be displayed on the device, and this process will be repeated until the witness is satisfied.

[1501] 6. Final confirmation and saving

[1502] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[1503] The user (police officer) selects "Save."

[1504] The server stores the final portrait data in a database.

[1505] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[1506] Specific examples

[1507] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[1508] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1509] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[1510] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[1511] 4. The server generates the first portrait based on the feature information and sends it to the device.

[1512] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1513] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[1514] 7. The server updates the portrait based on the correction data and sends it back to the device.

[1515] 8. Repeat this process until you have created a portrait that satisfies the witness.

[1516] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1517] The system allows witnesses to provide feedback in a relaxed manner, improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[1518] The processing flow will be explained below.

[1519] Step 1:

[1520] The terminal starts up and the officer enters their login information.

[1521] Step 2:

[1522] The server verifies the login information and authentication is complete.

[1523] Step 3:

[1524] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1525] Step 4:

[1526] A message will appear on the device indicating that the initial setup is complete.

[1527] Step 5:

[1528] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[1529] Step 6:

[1530] The user (police officer) inputs the obtained characteristic information into the terminal's UI.

[1531] Step 7:

[1532] The terminal transmits input data to the server in real time.

[1533] Step 8:

[1534] The server generates an initial portrait based on the received feature information.

[1535] Step 9:

[1536] The server transmits the generated portrait data to the terminal.

[1537] Step 10:

[1538] The first portrait will be displayed on the device.

[1539] Step 11:

[1540] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[1541] Step 12:

[1542] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[1543] Step 13:

[1544] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1545] Step 14:

[1546] The user (police officer) inputs the desired corrections into the device's UI.

[1547] Step 15:

[1548] The terminal transmits the correction data to the server.

[1549] Step 16:

[1550] The server updates the portrait based on the correction data.

[1551] Step 17:

[1552] The server transmits the updated portrait data to the terminal again.

[1553] Step 18:

[1554] The revised portrait will be displayed on the device.

[1555] Step 19:

[1556] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[1557] Step 20:

[1558] The users (witnesses and police officers) agree on the final sketch.

[1559] Step 21:

[1560] The user (police officer) selects "Save."

[1561] Step 22:

[1562] The server stores the final portrait data in a database.

[1563] Step 23:

[1564] The server notifies the terminal that the save is complete.

[1565] Step 24:

[1566] A message will appear on your device indicating that the save is complete.

[1567] Example 2

[1568] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1569] In conventional portrait generation systems, the input of information from witnesses and the correction of portraits are often cumbersome and time-consuming. Furthermore, witness tension and anxiety can affect the accuracy of the portrait. For this reason, there is a need to create an environment where witnesses can relax and provide detailed feedback. Furthermore, because it is difficult to generate and correct portraits in real time, there is a need for highly accurate and rapid investigative support.

[1570] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1571] In this invention, the server includes means for recognizing emotions from the facial expressions and voices of witnesses, means for dynamically adjusting a user interface based on the recognized emotion information, means for generating a caricature based on input feature information, and means for modifying the caricature based on input feedback. This allows the witness to provide feedback in a relaxed state, and enables the generation and modification of highly accurate caricatures in real time.

[1572] "Means for inputting characteristic information from witnesses" refers to an interface or device that allows witnesses to provide information about the appearance and characteristics of the perpetrator to the system.

[1573] "Means for generating a caricature based on input feature information" refers to an algorithm or program for automatically creating a caricature based on appearance and features input by a witness.

[1574] "Means for displaying the generated caricature to a witness" refers to an interface or device for showing the generated caricature to a witness through a display or monitor.

[1575] "Means for inputting feedback from witnesses" refers to an interface or device that allows witnesses to input their opinions and comments to the system for corrections and improvements to the generated portrait.

[1576] "Means for modifying the portrait based on input feedback" refers to an algorithm or program that automatically modifies the portrait based on feedback provided by witnesses.

[1577] "Means for displaying the modified portrait to the witness again" refers to an interface or device for displaying the modified portrait to the witness again through a display or monitor.

[1578] "Means for storing the final portrait in a database" refers to a system or program for storing the finalized portrait in a database in digital format.

[1579] "Means for recognizing emotions from the facial expressions and voice of witnesses" refers to emotion recognition algorithms or programs that analyze the facial expressions and voice of witnesses to identify their emotional state (e.g., tension, anxiety, relaxation) in real time.

[1580] "Means for dynamically adjusting a user interface based on recognized emotional information" refers to a system or program for automatically modifying or adjusting an interface in response to a recognized emotional state of a witness to allow the user to relax and provide feedback.

[1581] "Server" refers to a computer system that is a central processing unit for executing each of the above means and is responsible for major processes such as receiving, transmitting, and storing data, as well as generating and modifying portraits.

[1582] "Device" refers to the hardware device used by witnesses and police officers to input characteristic information and feedback and to display the sketch, including, for example, a computer, tablet, or smartphone.

[1583] An "emotion engine" refers to software or algorithms that analyze facial expressions and voice of witnesses to recognize their emotions and dynamically adjust the user interface or system behavior based on those emotions.

[1584] "User Interface (UI)" refers to the screen displays and controls used by witnesses and police officers to interact with the system, including the design and functionality to facilitate inputting information and providing feedback.

[1585] The present invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on feature information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[1586] System Configuration

[1587] The system consists of four main components:

[1588] 1. User Interface (UI): The screen through which witnesses and investigators interact.

[1589] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1590] 3. Terminal: An input device used by witnesses and police officers.

[1591] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[1592] Program processing

[1593] The server includes a means for inputting feature information from a witness, a means for generating a caricature based on the input feature information, a means for displaying the generated caricature to the witness, a means for inputting feedback from the witness, a means for modifying the caricature based on the feedback, a means for displaying the modified caricature again to the witness, and a means for saving the final caricature in a database. The system also includes an emotion engine for recognizing emotions from the facial expressions and voices of the witness, and a means for dynamically adjusting the UI based on the recognition results.

[1594] Hardware and Software

[1595] Hardware: The system components include a PC, a tablet, and a networked server.

[1596] Software: Includes emotion recognition algorithms, similar face generation algorithms, and database management systems.

[1597] Specific examples

[1598] As an example of implementation, we will explain a case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." Here, we will specifically explain the input process, emotion recognition process, caricature generation process, and correction process.

[1599] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1600] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[1601] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[1602] 4. The server generates the first portrait based on the feature information and sends it to the device.

[1603] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1604] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[1605] 7. The server updates the portrait based on the correction data and sends it back to the device.

[1606] 8. Repeat this process until you have created a portrait that satisfies the witness.

[1607] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1608] Prompt Sentence Examples

[1609] Below are some examples of specific prompts that can be input to the generative AI model in this system:

[1610] "A witness listed the suspect's characteristics as 'black hair, a round face, and narrow eyes.' Please generate an initial sketch based on this information."

[1611] "We have detected the witness' emotional state. They do not appear to be relaxed. Please adjust the user interface to help them relax."

[1612] "A witness pointed out a correction to the portrait, saying 'the eyes are narrower.' Please generate a new portrait that reflects this."

[1613] This invention allows witnesses to provide feedback in a relaxed state, improving the speed and accuracy of investigations and reducing the risk of false accusations.

[1614] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1615] Program processing flow

[1616] Step 1: Initial Setup

[1617] Step 1-1: Start the terminal and log in

[1618] Input: Police officer's login information (ID and password)

[1619] Output: Server authentication result

[1620] Specific operation:

[1621] The device will start up and display the login screen.

[1622] The user (police officer) enters their ID and password and clicks the "Login" button.

[1623] The terminal sends the input information to the server, which verifies the login information.

[1624] The server sends the authentication result to the terminal, and the terminal displays a message indicating "authentication successful" or "authentication failed."

[1625] Step 1-2: Establishing a database connection

[1626] Input: Server authentication success result

[1627] Output: Database connection established message

[1628] Specific operation:

[1629] Once the server receives the successful authentication message, it attempts to connect to the database.

[1630] The server initializes the caricature generation algorithm.

[1631] The server sends a message to the terminal indicating that the database connection has been successful, and the terminal displays the message "Initial settings completed."

[1632] Step 2: Hear from witnesses

[1633] Step 2-1: Interviewing witnesses

[1634] Input: Basic information of the witness (e.g., age, gender, hair color, etc.)

[1635] Output: Input data of eyewitness characteristics

[1636] Specific operation:

[1637] The user (police officer) asks witnesses questions about the basic characteristics of the perpetrator.

[1638] The user (police officer) enters the witness's answers into the device's UI.

[1639] Step 2-2: Real-time transmission of feature information

[1640] Input: Witness characteristics information entered into the terminal

[1641] Output: Feature information data sent to the server

[1642] Specific operation:

[1643] The device transmits the entered characteristic information of the witness to the server in real time.

[1644] The server receives the characteristic information and temporarily stores it in a database.

[1645] Step 3: Eyewitness emotion recognition

[1646] Step 3-1: Acquiring emotion data

[1647] Input: witness' facial expressions and voice data

[1648] Output: Recognized emotion information

[1649] Specific operation:

[1650] The emotion engine scans the facial expressions and voices of witnesses in real time via the device.

[1651] The emotion engine analyzes the collected data to recognize the witness's emotions (e.g., nervous, anxious, relaxed).

[1652] The emotion engine sends the recognition results to the server.

[1653] Step 3-2: Dynamic UI adjustment

[1654] Input: Recognized emotion information

[1655] Output: Adjusted user interface

[1656] Specific operation:

[1657] The server analyzes emotional information and determines UI settings that are easy to relax.

[1658] The server sends instructions to the terminal to change the UI settings, and the terminal dynamically adjusts the UI.

[1659] Step 4: Generate a caricature

[1660] Step 4-1: Generate the first portrait

[1661] Input: Witness characteristics information

[1662] Output: First portrait data

[1663] Specific operation:

[1664] The server runs an algorithm to generate an initial portrait based on the feature information.

[1665] The server transmits the generated portrait data to the terminal.

[1666] Step 4-2: Displaying the caricature

[1667] Input: First portrait data

[1668] Output: Initial portrait shown to witnesses

[1669] Specific operation:

[1670] The first portrait received by the device is displayed on the screen.

[1671] Step 5: Adjustments and feedback

[1672] Step 5-1: Input desired corrections

[1673] Input: Eyewitness feedback

[1674] Output: Data of desired correction points

[1675] Specific operation:

[1676] The user (witness) checks the sketch and tells the user (police officer) what needs to be corrected.

[1677] The user (police officer) inputs the desired corrections into the UI.

[1678] Step 5-2: Submitting the correction data

[1679] Input: Correction requests entered into the UI

[1680] Output: Corrected data sent to the server

[1681] Specific operation:

[1682] The terminal transmits the desired corrections to the server.

[1683] Step 5-3: Update your portrait

[1684] Input: Data of desired correction points

[1685] Output: Updated caricature data

[1686] Specific operation:

[1687] The server runs an algorithm to update the portrait based on the correction data.

[1688] The server transmits the updated portrait data to the terminal again.

[1689] Step 5-4: Displaying the updated caricature

[1690] Input: Updated caricature data

[1691] Output: Updated portrait displayed to witnesses

[1692] Specific operation:

[1693] The updated portrait received by the terminal is displayed on the display.

[1694] This process is repeated until the witness is satisfied.

[1695] Step 6: Final review and save

[1696] Step 6-1: Final confirmation

[1697] Input: Eyewitness confirmation of updated portrait

[1698] Output: Final caricature data

[1699] Specific operation:

[1700] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[1701] Step 6-2: Save the portrait

[1702] Input: Final portrait data

[1703] Output: Caricature data stored in a database

[1704] Specific operation:

[1705] The user (police officer) selects "Save."

[1706] The server stores the final portrait data in a database.

[1707] Step 6-3: Notification of save completion

[1708] Input: Database save completion result

[1709] Output: Message notifying you that saving is complete

[1710] Specific operation:

[1711] The server notifies the terminal that the save is complete, and the message "Saving complete" is displayed on the terminal.

[1712] (Application example 2)

[1713] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1714] In criminal investigations, conventional methods of generating sketches based on eyewitness testimony can lead to inaccurate testimony due to the eyewitness's emotions or nervousness, and can hinder smooth communication with investigators. Furthermore, generating and modifying sketches in real time is difficult, resulting in a lack of speed and accuracy in investigations. To solve these issues, a system is needed that can recognize eyewitness emotions and dynamically adjust the interface to generate and modify sketches.

[1715] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1716] In this invention, the server includes means for recognizing emotions from the facial expressions and voice of a witness, means for dynamically adjusting the interface based on the recognized emotion information, and means for generating a portrait based on the input feature information. This allows the witness to give accurate testimony in a relaxed state, and enables rapid and highly accurate generation of a portrait.

[1717] A "witness" is someone who directly witnessed an incident or accident, or was involved in it face-to-face.

[1718] "Characteristic information" refers to information that records details of the appearance and behavior of a person described by a witness.

[1719] A "caricature" is an illustration or image of a person's face based on eyewitness testimony.

[1720] "Facial expression" refers to temporary emotions and reactions expressed through facial muscle movements, gaze, etc.

[1721] "Voice" refers to the linguistic expressions such as tone, intonation, and volume of the voice spoken by the witness.

[1722] "Emotional information" refers to data that represents the psychological state of a witness analyzed by the emotion engine.

[1723] "Interface" refers to the screen and operating means for exchanging information between the user and the system.

[1724] "Feedback" refers to the behavior of witnesses expressing their opinions or requests for corrections regarding the generated portrait.

[1725] "Server" refers to a remote computer for storing, processing, and transmitting data.

[1726] This invention is a system that acquires feature information based on eyewitness testimony and uses an emotion engine to generate and modify highly accurate caricatures in real time. The system takes into account the emotions of the eyewitness and modifies the caricature as many times as necessary until the eyewitness is satisfied, and finally saves it in a database.

[1727] System programs and the hardware and software used

[1728] The system consists of the following main components:

[1729] 1. Server: Generates and modifies caricatures based on feature and emotion information.

[1730] Software used: Emotion recognition software, caricature generation algorithm

[1731] Hardware used: high-speed processor, database server

[1732] 2. Terminal: An input device used by witnesses or investigators to transmit input feature information and feedback information to the server. Examples include smart glasses, smartphones, and tablets.

[1733] Software used: Input interface, emotion recognition software

[1734] Hardware used: Smart glasses (e.g., Google Glass, Vuzix Blade), smartphone (iOS / Android)

[1735] 3. User Interface (UI): A screen for witness and investigator interaction, displaying the sketch generated in real time and providing the ability to input witness feedback.

[1736] Software used: Browser-based UI, desktop application, mobile app

[1737] Explanation of program processing

[1738] The server uses emotion recognition software to recognize emotions from the witness's facial expressions and voice. This software analyzes the witness's facial expression data and voice data to recognize their current emotional state (e.g., nervous, anxious, relaxed). Based on the emotional information, the server dynamically adjusts the interface to provide an environment in which the witness can relax and easily testify.

[1739] When feature information is entered on the device, it is sent to the server, which then generates an initial caricature based on the feature information using a caricature generation algorithm. The generated caricature is then displayed to the witness through a user interface. When the witness provides feedback, the device sends the information to the server, which then modifies the caricature based on the feedback.

[1740] This process is repeated until the witness is satisfied, and once the final portrait is completed, the server stores it in a database.

[1741] Specific examples

[1742] For example, if a witness says, "The perpetrator has black hair, a round face, and narrow eyes," these characteristics are entered into the device. The server uses emotion recognition software to analyze whether the witness is nervous or relaxed. If it detects nervousness, the server adjusts the user interface, changing the interface design and the questions presented to put the witness at ease.

[1743] After the first sketch is generated and displayed on the device, the witness may point out that the eyes are narrower. This feedback is sent to the server, which then modifies the sketch based on the feedback. This process is repeated several times until a final sketch is created that satisfies the witness.

[1744] Prompt Sentence Examples

[1745] "Please tell us the characteristics of the perpetrator. Please be as detailed as possible. What is the shape of the eyes? What color is the hair? What is the shape of the face? Please relax and answer."

[1746] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1747] Step 1:

[1748] The server initializes the system. The terminal starts up and the investigator enters their login information. Once the login information is entered, the server verifies it and performs authentication. Once authentication is complete, the server establishes a connection to the database and initializes the portrait generation algorithm. At this point, the terminal displays a message indicating that initialization is complete.

[1749] Input: Investigator login information

[1750] Output: Authentication complete message, initial setup complete message

[1751] Data processing / calculation: Authenticating login information, establishing database connections, initializing algorithms

[1752] Step 2:

[1753] The user (investigator) asks the witness for basic information (e.g., age, gender, hair color, etc.) in an interview format. The basic information provided by the witness is entered into the device's UI. The device then transmits the entered characteristic information to the server in real time.

[1754] Input: basic information and characteristics of the witness

[1755] Output: Sending the input feature information

[1756] Data processing / calculation: Input and organization of basic information, real-time transmission of characteristic information

[1757] Step 3:

[1758] The server receives the feature information and uses an emotion engine to recognize emotions from the witness's facial expressions and voice. The recognized emotion information is analyzed on the server side, and the UI is dynamically adjusted to make the witness feel relaxed and easy to give feedback.

[1759] Input: witness's facial expression data, voice data

[1760] Output: Recognized emotion information

[1761] Data processing / calculation: Analysis of facial expression data and voice data, emotion recognition, dynamic UI adjustment

[1762] Step 4:

[1763] The server generates an initial portrait based on the received feature information. A portrait generation algorithm is used for this generation. The generated portrait data is sent to the terminal, and the initial portrait is displayed on the terminal.

[1764] Input: feature information

[1765] Output: Sending and displaying the initial portrait data

[1766] Data processing / calculation: Generate portraits based on feature information, send portrait data

[1767] Step 5:

[1768] The user (witness) checks the sketch and tells the investigator what they would like to be corrected (e.g., "Make the eyes thinner"). The investigator inputs the witness's feedback into the device's UI. The input correction requests are sent from the device to the server.

[1769] Input: Witness feedback, correction requests

[1770] Output: Sending correction requests

[1771] Data processing / calculation: Input and sorting of feedback, sending correction requests

[1772] Step 6:

[1773] The server then modifies the portrait based on the requested corrections. The modified portrait data is then sent back to the terminal and displayed on the device. This process is repeated until the witness is satisfied.

[1774] Input: desired corrections

[1775] Output: Send and display the corrected portrait data

[1776] Data processing / calculation: Correction of portraits based on desired correction points, resending of portrait data

[1777] Step 7:

[1778] The users (witness and investigator) review the final sketch and, if satisfied, agree on it as the final sketch. The investigator selects "Save," and the server saves the final sketch data in the database. A notification that saving is complete is displayed on the device.

[1779] Input: Final portrait data

[1780] Output: Database save, save completion notification

[1781] Data processing / calculation: Saving portrait data and sending a notification when saving is complete

[1782] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1783] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1784] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1785] [Fourth embodiment]

[1786] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1787] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1788] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1789] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1790] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1791] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1792] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1793] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1794] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1795] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1796] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1797] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1798] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1799] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[1800] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[1801] System Configuration

[1802] The system consists of three main components:

[1803] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[1804] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1805] 3. Terminal: An input device used by witnesses and police officers.

[1806] Program processing overview

[1807] 1. Initial Setup

[1808] The terminal starts up and the officer logs in.

[1809] The server verifies the login information and authentication is complete.

[1810] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1811] A message will appear on the device indicating that the initial setup is complete.

[1812] 2. Hearing from witnesses

[1813] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[1814] The terminal transmits input data to the server in real time.

[1815] 3. Caricature Generation

[1816] The server generates an initial portrait based on the received feature information.

[1817] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[1818] 4. Adjustments and feedback

[1819] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1820] The user (police officer) inputs the desired corrections into the device's UI.

[1821] The terminal transmits the correction data to the server.

[1822] The server updates the portrait based on the correction data, and transmits the updated portrait data to the terminal again.

[1823] The updated portrait will be displayed on the device, and the process will be repeated until the witness is satisfied.

[1824] 5. Final confirmation and saving

[1825] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[1826] The user (police officer) selects "Save."

[1827] The server stores the final portrait data in a database.

[1828] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[1829] Specific examples

[1830] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[1831] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1832] 2. The server generates the first portrait based on the feature information and sends it to the device.

[1833] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1834] 4. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[1835] 5. The server updates the portrait based on the correction data and sends it back to the device.

[1836] 6. Repeat this process until you have created a portrait that satisfies the witness.

[1837] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1838] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[1839] The processing flow will be explained below.

[1840] Step 1:

[1841] The terminal starts up and the officer enters their login information.

[1842] Step 2:

[1843] The server verifies the login information and authentication is complete.

[1844] Step 3:

[1845] The server establishes a connection to the database and initializes the caricature generation algorithm.

[1846] Step 4:

[1847] A message will appear on the device indicating that the initial setup is complete.

[1848] Step 5:

[1849] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[1850] Step 6:

[1851] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[1852] Step 7:

[1853] The terminal transmits the input characteristic information to the server in real time.

[1854] Step 8:

[1855] The server generates an initial portrait based on the received feature information.

[1856] Step 9:

[1857] The server transmits the generated portrait data to the terminal.

[1858] Step 10:

[1859] The first portrait will be displayed on the device.

[1860] Step 11:

[1861] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[1862] Step 12:

[1863] The user (police officer) inputs the desired corrections into the device's UI.

[1864] Step 13:

[1865] The terminal transmits the correction data to the server.

[1866] Step 14:

[1867] The server updates the portrait based on the correction data.

[1868] Step 15:

[1869] The server transmits the updated portrait data to the terminal again.

[1870] Step 16:

[1871] The revised portrait will be displayed on the device.

[1872] Step 17:

[1873] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[1874] Step 18:

[1875] The users (witnesses and police officers) agree on the final sketch.

[1876] Step 19:

[1877] The user (police officer) selects "Save."

[1878] Step 20:

[1879] The server stores the final portrait data in a database.

[1880] Step 21:

[1881] The server notifies the terminal that the save is complete.

[1882] Step 22:

[1883] A message will appear on your device indicating that the save is complete.

[1884] Example 1

[1885] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1886] In criminal investigations, the task of creating a portrait based on eyewitness testimony is time-consuming and labor-intensive. Furthermore, eyewitnesses' memories are often vague, requiring repeated revisions, making it difficult to quickly generate a highly accurate portrait. To address these challenges, a technology is needed that can generate and revise portraits in real time based on information obtained from eyewitnesses, ultimately resulting in a highly accurate portrait.

[1887] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1888] In this invention, the server includes a means for a user to input feature information from a witness, a means for generating a portrait using a generative AI model based on the input feature information, and a means for displaying the corrected portrait to the user again. This allows for the generation and correction of portraits based on witness testimony in real time, thereby improving the efficiency and accuracy of investigations.

[1889] The "user" is the person who operates the system, inputs information from witnesses, and reviews the generated and modified portraits.

[1890] A "witness" is someone who saw a suspect at a crime scene or elsewhere, and whose testimony is used to generate or modify the sketch.

[1891] "Characteristic information" refers to specific physical characteristics of a suspect as testified by witnesses (e.g., age, gender, hair color, face shape, eye shape, etc.).

[1892] The "server" is the central computing device of the system, a combination of hardware and software that receives feature information, generates and modifies portraits using generative AI models, and stores data.

[1893] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate and modify caricatures in real time using feature information as input.

[1894] A "caricature" is an image of a suspect's face generated from witness testimony and can be digitally displayed and modified.

[1895] "Feedback" is information including desired corrections and suggestions made by the witness to the generated portrait.

[1896] A "database" is a digital storage system that organizes and stores similar or related data and allows it to be retrieved as needed.

[1897] "Real-time" refers to a state in which data is generated or modified immediately and the results are reflected immediately to the user.

[1898] "Input means" refers to an interface or device that allows a user to input feature information and feedback to the system.

[1899] The "display means" refers to a screen or monitor that visually presents the generated portrait and the correction results to the user.

[1900] This invention relates to a system that supports criminal investigations with high accuracy and speed by generating a portrait in real time based on characteristic information obtained from witnesses and modifying it as necessary.

[1901] This system involves a series of processes for acquiring characteristic information from witnesses and generating and modifying portraits. The system configuration and specific program processing are explained below.

[1902] System Configuration

[1903] The system consists of three main components:

[1904] 1. User Interface (UI): The screen through which witnesses and users interact.

[1905] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[1906] 3. Terminal: An input device used by witnesses and users.

[1907] The system operates as follows.

[1908] 1. Initial Setup

[1909] The terminal starts up and the user (police officer) logs in. The server verifies the login information and authentication is completed. The server establishes a connection to the database and initializes the caricature generation algorithm. A message indicating initial setup is complete is displayed on the terminal.

[1910] 2. Hearing from witnesses

[1911] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the information into the device's UI. The device then sends the input data to the server in real time.

[1912] 3. Caricature Generation

[1913] The server generates an initial portrait based on the received feature information, and then transmits the generated portrait data to the terminal, where the initial portrait is displayed.

[1914] 4. Adjustments and feedback

[1915] The user (witness) checks the sketch and tells the user (police officer) what corrections they would like to make (e.g., "Make the eyes thinner"). The user (police officer) enters the corrections they would like to make into the device's UI. The device sends the correction data to the server. The server updates the sketch based on the correction data, and the server sends the updated sketch data back to the device. The updated sketch is displayed on the device. This process is repeated until the witness is satisfied.

[1916] 5. Final confirmation and saving

[1917] The users (witness and police officer) review the final sketch, and if they are satisfied, they agree on it as the final sketch. The user (police officer) selects "Save." The server saves the final sketch data in the database. The server notifies the device that saving is complete, and a message indicating this is displayed on the device.

[1918] Specific examples

[1919] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[1920] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[1921] 2. The server generates the first portrait based on the feature information and sends it to the device.

[1922] 3. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[1923] 4. The user (police officer) inputs the correction, such as "the eyes should be narrower," and the device sends the correction information to the server.

[1924] 5. The server updates the portrait based on the correction data and sends it back to the device.

[1925] 6. Repeat this process until you have created a portrait that satisfies the witness.

[1926] 7. The user (police officer) selects "Save" and the server saves the sketch in the database.

[1927] The system allows for real-time adjustments to the sketch until the witness is satisfied, thereby improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[1928] Prompt Sentence Examples

[1929] markdown

[1930] This AI model is used to generate a portrait based on witness testimony, which includes the following:

[1931] Hair color: Black

[1932] Face Shape: Round

[1933] Eye shape: Narrow

[1934] Use these features to generate an initial caricature, then follow the instructions to make any necessary corrections.

[1935] In this way, this system uses artificial intelligence technology to quickly and accurately convert eyewitness testimony into a facial sketch, dramatically improving the efficiency of criminal investigations.

[1936] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1937] Step 1: Initial Setup

[1938] The terminal boots up and the user accesses a login screen, which takes a username and password as input.

[1939] The server receives the username and password and verifies the authentication information using an SQL database. The authentication information is verified as part of the data processing, and the authentication result (success / failure) is obtained as the output.

[1940] If the server is successful in authentication, it establishes a connection to the database and initializes the caricature generation algorithm. The output is the connection status and the initialization completion status of the algorithm.

[1941] A message indicating that initial setup is complete is displayed on the terminal, and the user confirms that the system is ready.

[1942] Step 2: Interview witnesses

[1943] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and obtains characteristic information from the witness as input.

[1944] The user (police officer) inputs the feature information into the device's UI. The feature information is received as input data.

[1945] The terminal transmits the input characteristic information to the server in real time. The characteristic information is received as data processing, and the characteristic information is output and transmitted to the server.

[1946] Step 3: Generate a caricature

[1947] The server receives the feature information sent from the device and inputs it into the generative AI model, using the feature information as input data.

[1948] The server uses the generative AI model to generate an initial portrait based on the feature information. The generation process is carried out as a data calculation, and the generated portrait data is obtained as the output.

[1949] The server transmits the generated portrait data to the terminal.

[1950] The device displays the sketch on the screen, and the user (police officer) asks a witness to confirm the sketch.

[1951] Step 4: Adjustments and feedback

[1952] The user (witness) checks the generated sketch and tells the police officer what they would like to improve (e.g., "Make the eyes thinner"). Feedback information is received as input.

[1953] The user (police officer) inputs the desired corrections into the device's UI. The correction information is received as input data.

[1954] The terminal transmits the correction data to the server. The correction information is received as data processing, and the correction data is output and transmitted to the server.

[1955] The server re-executes the generative AI model based on the received correction data, and updates the portrait. The update process is performed as data calculation, and the updated portrait data is obtained as output.

[1956] The server transmits the updated portrait data to the terminal.

[1957] The device displays the updated sketch on the screen, and the user (police officer) asks the witness to review the updated sketch. This process is repeated until the witness is satisfied.

[1958] Step 5: Final review and save

[1959] The users (witnesses and police officers) review the final sketch and, if they are satisfied, agree on it as the final version.,Final confirmation information is received as input.

[1960] The user (police officer) selects "Save." A save instruction is received as an operational input.

[1961] The server saves the final portrait data in the database. The data saving process is performed as data processing, and the saving completion status is obtained as output.

[1962] The server notifies the terminal that the save is complete.

[1963] The terminal displays a message indicating that the data has been saved, and the process ends. The user (police officer) confirms that the process has been completed successfully.

[1964] In this way, through the specific actions and data processing performed at each step, the system can generate and modify highly accurate portraits in real time based on eyewitness testimony.

[1965] (Application example 1)

[1966] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1967] Traditional criminal investigation sketch creation processes are often manual, making it difficult to quickly and accurately generate sketches. Furthermore, due to a lack of a mechanism for incorporating real-time feedback from witnesses, it is difficult to quickly create a final sketch that satisfies the witnesses. This makes it difficult to effectively respond in situations where speed and accuracy are required in investigations.

[1968] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1969] In this invention, the server includes means for inputting feature information from a witness, means for generating a portrait based on the input feature information, means for displaying the generated portrait to the witness, means for inputting feedback from the witness, means for modifying the portrait based on the input feedback, means for displaying the modified portrait again to the witness, means for saving the final portrait in a database, means for transmitting and receiving feature information to the server in real time using a smart device, means for transmitting and receiving feedback to the server in real time, means for inputting feature information and feedback using prompt sentences, and means for generating and modifying a portrait based on the feature information and feedback using a generative AI model, thereby enabling rapid and highly accurate generation and modification of portraits.

[1970] "Characteristic information" is information provided by witnesses that indicates the criminal's external features and characteristics.

[1971] A "portrait" is a drawing that represents an image of the criminal's face, generated based on feature information.

[1972] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate and modify caricatures based on feature information and feedback.

[1973] A "prompt" is a phrase that guides witnesses and police officers when entering characteristic information and feedback.

[1974] "Feedback" is information indicating corrections or requests for improvement provided by a witness to the displayed portrait.

[1975] "Smart devices" refer to portable electronic devices with advanced functions, such as smartphones and tablets.

[1976] "Server" refers to a central processing unit that receives feature information and feedback, generates and modifies caricatures, and stores them in a database.

[1977] An "eyewitness" is someone who actually witnesses a crime scene or the circumstances of an incident and provides testimony.

[1978] The "database" is a data storage system for permanently storing the generated portraits and related information.

[1979] An "input device" is a hardware device used by a witness or police officer to input characteristic information or feedback.

[1980] This invention is a system that quickly generates a portrait based on characteristic information obtained from witnesses and makes corrections as necessary. This system generates a portrait in real time based on witness testimony, supporting investigations with high accuracy and efficiency.

[1981] System configuration

[1982] The system of the present invention comprises the following main components:

[1983] 1. User Interface (UI): A screen for interaction between witnesses and police officers. Implemented on smart devices (smartphones and tablets), it includes a testimony input form, a portrait display area, adjustment buttons, etc.

[1984] 2. Server: A central processing unit that processes caricature generation. It receives feature information and feedback, and generates and modifies caricatures using generative AI models.

[1985] 3. Database: A storage system for storing the generated portraits and related information.

[1986] Program processing overview

[1987] Hardware and software used

[1988] Hardware: Smart devices (smartphones, tablets), servers

[1989] Software: React Native or Swift UI (user interface), Flask (backend), TensorFlow (generative AI models), MySQL or MongoDB (database)

[1990] Data processing and calculation

[1991] Input and transmission of characteristic information: The user inputs the characteristic information of the witness through the UI on the smart device, and this information is transmitted to the server in real time.

[1992] Caricature generation: The server generates a caricature using TensorFlow based on the received feature information. The generated caricature is sent back to the smart device and displayed on the UI.

[1993] Feedback acquisition and revision: The witness enters feedback (requests for revision) on the portrait, which is then sent back to the server. The server then revises the portrait based on the feedback and sends the updated portrait back to the smart device. This process is repeated until the witness is satisfied.

[1994] Final saving: Once a final satisfactory caricature is generated, it is saved in the database.

[1995] Specific examples

[1996] For example, consider the case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." The user inputs characteristics such as "black hair," "round face," and "narrow eyes" into the UI of their smart device, which is then sent to the server. The server generates an initial sketch based on this characteristic information and sends it back to the smart device. If the witness provides feedback such as "his eyes are a little narrower," the sketch is sent again to the server, and the sketch is revised and reviewed by the witness. This process is repeated until the witness is satisfied, and a satisfactory sketch is generated and saved.

[1997] Prompt Sentence Examples

[1998] Text prompt:

[1999] TXT

[2000] Testimony input prompt:

[2001] Please enter the characteristics information provided by the witness in the following format:

[2002] Name: [Witness A]

[2003] Age: [Around 30 years old]

[2004] Hair color: [Black]

[2005] Eye shape: [narrow]

[2006] Once you have completed the input, please press the "Submit" button.

[2007] Caricature correction prompt:

[2008] Please review your current portrait and enter any necessary corrections in the following format:

[2009] Correction: [Eyes are a little narrower]

[2010] Reason for revision: [To make the impression of the actual perpetrator more realistic]

[2011] Once you have completed the entry, press the "Edit" button.

[2012] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2013] Step 1:

[2014] The terminal starts up and the officer logs in.

[2015] Input: Police officer's login information (username, password)

[2016] Operation: The device sends login information to the server. The server verifies the login information and performs authentication. If authentication is successful, the device displays a message indicating initial setup is complete.

[2017] Output: A notification that initial setup is complete will be displayed on the device.

[2018] Step 2:

[2019] The user (police officer) asks the witness for basic information (e.g., age, gender, hair color, etc.) and enters the obtained information into the device's UI.

[2020] Input: Witness characteristics (e.g., age, gender, hair color, face shape, eye shape)

[2021] Operation: The device sends the input feature information to the server in real time.

[2022] Output: The feature information is sent to the server for processing.

[2023] Step 3:

[2024] The server generates an initial portrait based on the received feature information.

[2025] Input: feature information

[2026] How it works: The generative AI model on the server generates a portrait based on feature information.

[2027] Output: The generated portrait data is created and sent to the device.

[2028] Step 4:

[2029] The initial portrait is displayed on the device, and the user (witness) checks it.

[2030] Input: Generated caricature data

[2031] How it works: The device displays a sketch to a witness, who reviews it and tells the police officer what corrections they would like to make.

[2032] Output: The witness' desired correction score is obtained.

[2033] Step 5:

[2034] The user (police officer) inputs the desired corrections (e.g., "the eyes should be narrower") into the device's UI.

[2035] Input: Witness' desired correction point

[2036] Operation: The device sends the desired corrections to the server.

[2037] Output: The desired correction points are sent to the server for processing.

[2038] Step 6:

[2039] The server updates the portrait based on the desired corrections.

[2040] Input: desired corrections

[2041] How it works: The generative AI model on the server modifies the portrait, taking into account the desired corrections.

[2042] Output: The corrected portrait data is created and sent to the device.

[2043] Step 7:

[2044] The updated portrait is displayed on the device, and the user (witness) checks it again.

[2045] Input: Corrected portrait data

[2046] How it works: The device displays the revised portrait to the witness. If the witness is not satisfied, the process returns to input further revisions.

[2047] Output: The witness's desired correction points are obtained, or the final portrait is confirmed by the witness.

[2048] Step 8:

[2049] The user (police officer) checks the final portrait and selects "Save" if they are satisfied.

[2050] Input: Final caricature approval

[2051] Operation: The device sends a "save" request to the server, which saves the final portrait data to a database.

[2052] Output: A notification that the final portrait has been saved will be displayed on your device.

[2053] By following these processing steps, a system is realized that can quickly and accurately generate and modify caricatures in real time.

[2054] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2055] This invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on characteristic information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[2056] This system includes a series of processes for acquiring feature information from witnesses and generating and modifying caricatures, as well as a function for recognizing the emotions of witnesses and dynamically adjusting the interface and generation process accordingly. The system configuration and specific program processing are explained below.

[2057] System Configuration

[2058] The system consists of four main components:

[2059] 1. User Interface (UI): The screen through which witnesses and police officers interact.

[2060] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[2061] 3. Terminal: An input device used by witnesses and police officers.

[2062] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[2063] Program processing overview

[2064] 1. Initial Setup

[2065] The terminal starts up and the officer enters their login information.

[2066] The server verifies the login information and authentication is complete.

[2067] The server establishes a connection to the database and initializes the caricature generation algorithm.

[2068] A message will appear on the device indicating that the initial setup is complete.

[2069] 2. Hearing from witnesses

[2070] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[2071] The user (police officer) inputs characteristic information obtained from witnesses into the device's UI.

[2072] The terminal transmits input data to the server in real time.

[2073] 3. Eyewitness emotion recognition

[2074] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[2075] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[2076] 4. Caricature Generation

[2077] The server generates an initial portrait based on the received feature information.

[2078] The server transmits the generated portrait data to the terminal, and the initial portrait is displayed on the terminal.

[2079] 5. Adjustments and Feedback

[2080] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[2081] The user (police officer) inputs the desired corrections into the device's UI.

[2082] The terminal transmits the correction data to the server.

[2083] The server updates the portrait based on the correction data and sends it back to the device.

[2084] The corrected portrait will be displayed on the device, and this process will be repeated until the witness is satisfied.

[2085] 6. Final confirmation and saving

[2086] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[2087] The user (police officer) selects "Save."

[2088] The server stores the final portrait data in a database.

[2089] The server notifies the terminal that the save is complete, and a message indicating this is displayed on the terminal.

[2090] Specific examples

[2091] For example, consider a case where a witness testifies that the perpetrator has black hair, a round face, and narrow eyes.

[2092] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[2093] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[2094] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[2095] 4. The server generates the first portrait based on the feature information and sends it to the device.

[2096] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[2097] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[2098] 7. The server updates the portrait based on the correction data and sends it back to the device.

[2099] 8. Repeat this process until you have created a portrait that satisfies the witness.

[2100] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[2101] The system allows witnesses to provide feedback in a relaxed manner, improving the speed and accuracy of investigations and reducing the risk of wrongful convictions.

[2102] The processing flow will be explained below.

[2103] Step 1:

[2104] The terminal starts up and the officer enters their login information.

[2105] Step 2:

[2106] The server verifies the login information and authentication is complete.

[2107] Step 3:

[2108] The server establishes a connection to the database and initializes the caricature generation algorithm.

[2109] Step 4:

[2110] A message will appear on the device indicating that the initial setup is complete.

[2111] Step 5:

[2112] The user (police officer) interviews the witness and asks for basic information (e.g., age, gender, hair color, etc.).

[2113] Step 6:

[2114] The user (police officer) inputs the obtained characteristic information into the terminal's UI.

[2115] Step 7:

[2116] The terminal transmits input data to the server in real time.

[2117] Step 8:

[2118] The server generates an initial portrait based on the received feature information.

[2119] Step 9:

[2120] The server transmits the generated portrait data to the terminal.

[2121] Step 10:

[2122] The first portrait will be displayed on the device.

[2123] Step 11:

[2124] The emotion engine recognizes emotions from the witness's facial expressions and voice and sends that information to the server.

[2125] Step 12:

[2126] The server analyzes emotional information and dynamically adjusts the UI to make the witness feel relaxed and give feedback easily.

[2127] Step 13:

[2128] The user (witness) checks the sketch and tells the user (police officer) what they would like to change (e.g., "make the eyes narrower").

[2129] Step 14:

[2130] The user (police officer) inputs the desired corrections into the device's UI.

[2131] Step 15:

[2132] The terminal transmits the correction data to the server.

[2133] Step 16:

[2134] The server updates the portrait based on the correction data.

[2135] Step 17:

[2136] The server transmits the updated portrait data to the terminal again.

[2137] Step 18:

[2138] The revised portrait will be displayed on the device.

[2139] Step 19:

[2140] The user (witness) checks the corrections and requests additional corrections if necessary. This process is repeated until the witness is satisfied.

[2141] Step 20:

[2142] The users (witnesses and police officers) agree on the final sketch.

[2143] Step 21:

[2144] The user (police officer) selects "Save."

[2145] Step 22:

[2146] The server stores the final portrait data in a database.

[2147] Step 23:

[2148] The server notifies the terminal that the save is complete.

[2149] Step 24:

[2150] A message will appear on your device indicating that the save is complete.

[2151] Example 2

[2152] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2153] In conventional portrait generation systems, the input of information from witnesses and the correction of portraits are often cumbersome and time-consuming. Furthermore, witness tension and anxiety can affect the accuracy of the portrait. For this reason, there is a need to create an environment where witnesses can relax and provide detailed feedback. Furthermore, because it is difficult to generate and correct portraits in real time, there is a need for highly accurate and rapid investigative support.

[2154] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2155] In this invention, the server includes means for recognizing emotions from the facial expressions and voices of witnesses, means for dynamically adjusting a user interface based on the recognized emotion information, means for generating a caricature based on input feature information, and means for modifying the caricature based on input feedback. This allows the witness to provide feedback in a relaxed state, and enables the generation and modification of highly accurate caricatures in real time.

[2156] "Means for inputting characteristic information from witnesses" refers to an interface or device that allows witnesses to provide information about the appearance and characteristics of the perpetrator to the system.

[2157] "Means for generating a caricature based on input feature information" refers to an algorithm or program for automatically creating a caricature based on appearance and features input by a witness.

[2158] "Means for displaying the generated caricature to a witness" refers to an interface or device for showing the generated caricature to a witness through a display or monitor.

[2159] "Means for inputting feedback from witnesses" refers to an interface or device that allows witnesses to input their opinions and comments to the system for corrections and improvements to the generated portrait.

[2160] "Means for modifying the portrait based on input feedback" refers to an algorithm or program that automatically modifies the portrait based on feedback provided by witnesses.

[2161] "Means for displaying the modified portrait to the witness again" refers to an interface or device for displaying the modified portrait to the witness again through a display or monitor.

[2162] "Means for storing the final portrait in a database" refers to a system or program for storing the finalized portrait in a database in digital format.

[2163] "Means for recognizing emotions from the facial expressions and voice of witnesses" refers to emotion recognition algorithms or programs that analyze the facial expressions and voice of witnesses to identify their emotional state (e.g., tension, anxiety, relaxation) in real time.

[2164] "Means for dynamically adjusting a user interface based on recognized emotional information" refers to a system or program for automatically modifying or adjusting an interface in response to a recognized emotional state of a witness to allow the user to relax and provide feedback.

[2165] "Server" refers to a computer system that is a central processing unit for executing each of the above means and is responsible for major processes such as receiving, transmitting, and storing data, as well as generating and modifying portraits.

[2166] "Device" refers to the hardware device used by witnesses and police officers to input characteristic information and feedback and to display the sketch, including, for example, a computer, tablet, or smartphone.

[2167] An "emotion engine" refers to software or algorithms that analyze facial expressions and voice of witnesses to recognize their emotions and dynamically adjust the user interface or system behavior based on those emotions.

[2168] "User Interface (UI)" refers to the screen displays and controls used by witnesses and police officers to interact with the system, including the design and functionality to facilitate inputting information and providing feedback.

[2169] The present invention relates to a system that combines an emotion engine to support criminal investigations quickly and with high accuracy by generating portraits in real time based on feature information obtained from witnesses and modifying them as necessary while taking into account the emotions of the witnesses.

[2170] System Configuration

[2171] The system consists of four main components:

[2172] 1. User Interface (UI): The screen through which witnesses and investigators interact.

[2173] 2. Server: Responsible for operating the database and algorithms that process the caricature generation.

[2174] 3. Terminal: An input device used by witnesses and police officers.

[2175] 4. Emotion engine: Recognizes emotions from the witness's facial expressions and voice and provides that information to the system.

[2176] Program processing

[2177] The server includes a means for inputting feature information from a witness, a means for generating a caricature based on the input feature information, a means for displaying the generated caricature to the witness, a means for inputting feedback from the witness, a means for modifying the caricature based on the feedback, a means for displaying the modified caricature again to the witness, and a means for saving the final caricature in a database. The system also includes an emotion engine for recognizing emotions from the facial expressions and voices of the witness, and a means for dynamically adjusting the UI based on the recognition results.

[2178] Hardware and Software

[2179] Hardware: The system components include a PC, a tablet, and a networked server.

[2180] Software: Includes emotion recognition algorithms, similar face generation algorithms, and database management systems.

[2181] Specific examples

[2182] As an example of implementation, we will explain a case where a witness testifies that "the criminal has black hair, a round face, and narrow eyes." Here, we will specifically explain the input process, emotion recognition process, caricature generation process, and correction process.

[2183] 1. The user (police officer) enters characteristics such as "black hair," "round face," and "narrow eyes" into the device's UI, and the device sends the characteristic information to the server.

[2184] 2. The emotion engine detects tension and anxiety from the witness's facial expressions and voice and sends the information to the server.

[2185] 3. The server adjusts the UI based on emotional information, providing an environment where witnesses can relax and easily give feedback.

[2186] 4. The server generates the first portrait based on the feature information and sends it to the device.

[2187] 5. The first sketch is displayed on the device, and the witness points out that the eyes are narrower.

[2188] 6. The user (police officer) inputs the correction, such as "narrow eyes," and the device sends the correction information to the server.

[2189] 7. The server updates the portrait based on the correction data and sends it back to the device.

[2190] 8. Repeat this process until you have created a portrait that satisfies the witness.

[2191] 9. The user (police officer) selects "Save" and the server saves the sketch in the database.

[2192] Prompt Sentence Examples

[2193] Below are some examples of specific prompts that can be input to the generative AI model in this system:

[2194] "A witness listed the suspect's characteristics as 'black hair, a round face, and narrow eyes.' Please generate an initial sketch based on this information."

[2195] "We have detected the witness' emotional state. They do not appear to be relaxed. Please adjust the user interface to help them relax."

[2196] "A witness pointed out a correction to the portrait, saying 'the eyes are narrower.' Please generate a new portrait that reflects this."

[2197] This invention allows witnesses to provide feedback in a relaxed state, improving the speed and accuracy of investigations and reducing the risk of false accusations.

[2198] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2199] Program processing flow

[2200] Step 1: Initial Setup

[2201] Step 1-1: Start the terminal and log in

[2202] Input: Police officer's login information (ID and password)

[2203] Output: Server authentication result

[2204] Specific operation:

[2205] The device will start up and display the login screen.

[2206] The user (police officer) enters their ID and password and clicks the "Login" button.

[2207] The terminal sends the input information to the server, which verifies the login information.

[2208] The server sends the authentication result to the terminal, and the terminal displays a message indicating "authentication successful" or "authentication failed."

[2209] Step 1-2: Establishing a database connection

[2210] Input: Server authentication success result

[2211] Output: Database connection established message

[2212] Specific operation:

[2213] Once the server receives the successful authentication message, it attempts to connect to the database.

[2214] The server initializes the caricature generation algorithm.

[2215] The server sends a message to the terminal indicating that the database connection has been successful, and the terminal displays the message "Initial settings completed."

[2216] Step 2: Hear from witnesses

[2217] Step 2-1: Interviewing witnesses

[2218] Input: Basic information of the witness (e.g., age, gender, hair color, etc.)

[2219] Output: Input data of eyewitness characteristics

[2220] Specific operation:

[2221] The user (police officer) asks witnesses questions about the basic characteristics of the perpetrator.

[2222] The user (police officer) enters the witness's answers into the device's UI.

[2223] Step 2-2: Real-time transmission of feature information

[2224] Input: Witness characteristics information entered into the terminal

[2225] Output: Feature information data sent to the server

[2226] Specific operation:

[2227] The device transmits the entered characteristic information of the witness to the server in real time.

[2228] The server receives the characteristic information and temporarily stores it in a database.

[2229] Step 3: Eyewitness emotion recognition

[2230] Step 3-1: Acquiring emotion data

[2231] Input: witness' facial expressions and voice data

[2232] Output: Recognized emotion information

[2233] Specific operation:

[2234] The emotion engine scans the facial expressions and voices of witnesses in real time via the device.

[2235] The emotion engine analyzes the collected data to recognize the witness's emotions (e.g., nervous, anxious, relaxed).

[2236] The emotion engine sends the recognition results to the server.

[2237] Step 3-2: Dynamic UI adjustment

[2238] Input: Recognized emotion information

[2239] Output: Adjusted user interface

[2240] Specific operation:

[2241] The server analyzes emotional information and determines UI settings that are easy to relax.

[2242] The server sends instructions to the terminal to change the UI settings, and the terminal dynamically adjusts the UI.

[2243] Step 4: Generate a caricature

[2244] Step 4-1: Generate the first portrait

[2245] Input: Witness characteristics information

[2246] Output: First portrait data

[2247] Specific operation:

[2248] The server runs an algorithm to generate an initial portrait based on the feature information.

[2249] The server transmits the generated portrait data to the terminal.

[2250] Step 4-2: Displaying the caricature

[2251] Input: First portrait data

[2252] Output: Initial portrait shown to witnesses

[2253] Specific operation:

[2254] The first portrait received by the device is displayed on the screen.

[2255] Step 5: Adjustments and feedback

[2256] Step 5-1: Input desired corrections

[2257] Input: Eyewitness feedback

[2258] Output: Data of desired correction points

[2259] Specific operation:

[2260] The user (witness) checks the sketch and tells the user (police officer) what needs to be corrected.

[2261] The user (police officer) inputs the desired corrections into the UI.

[2262] Step 5-2: Submitting the correction data

[2263] Input: Correction requests entered into the UI

[2264] Output: Corrected data sent to the server

[2265] Specific operation:

[2266] The terminal transmits the desired corrections to the server.

[2267] Step 5-3: Update your portrait

[2268] Input: Data of desired correction points

[2269] Output: Updated caricature data

[2270] Specific operation:

[2271] The server runs an algorithm to update the portrait based on the correction data.

[2272] The server transmits the updated portrait data to the terminal again.

[2273] Step 5-4: Displaying the updated caricature

[2274] Input: Updated caricature data

[2275] Output: Updated portrait displayed to witnesses

[2276] Specific operation:

[2277] The updated portrait received by the terminal is displayed on the display.

[2278] This process is repeated until the witness is satisfied.

[2279] Step 6: Final review and save

[2280] Step 6-1: Final confirmation

[2281] Input: Eyewitness confirmation of updated portrait

[2282] Output: Final caricature data

[2283] Specific operation:

[2284] The users (witnesses and police officers) review the final sketch, and if they are satisfied, they agree on it as the final sketch.

[2285] Step 6-2: Save the portrait

[2286] Input: Final portrait data

[2287] Output: Caricature data stored in a database

[2288] Specific operation:

[2289] The user (police officer) selects "Save."

[2290] The server stores the final portrait data in a database.

[2291] Step 6-3: Notification of save completion

[2292] Input: Database save completion result

[2293] Output: Message notifying you that saving is complete

[2294] Specific operation:

[2295] The server notifies the terminal that the save is complete, and the message "Saving complete" is displayed on the terminal.

[2296] (Application example 2)

[2297] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2298] In criminal investigations, conventional methods of generating sketches based on eyewitness testimony can lead to inaccurate testimony due to the eyewitness's emotions or nervousness, and can hinder smooth communication with investigators. Furthermore, generating and modifying sketches in real time is difficult, resulting in a lack of speed and accuracy in investigations. To solve these issues, a system is needed that can recognize eyewitness emotions and dynamically adjust the interface to generate and modify sketches.

[2299] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2300] In this invention, the server includes means for recognizing emotions from the facial expressions and voice of a witness, means for dynamically adjusting the interface based on the recognized emotion information, and means for generating a portrait based on the input feature information. This allows the witness to give accurate testimony in a relaxed state, and enables rapid and highly accurate generation of a portrait.

[2301] A "witness" is someone who directly witnessed an incident or accident, or was involved in it face-to-face.

[2302] "Characteristic information" refers to information that records details of the appearance and behavior of a person described by a witness.

[2303] A "caricature" is an illustration or image of a person's face based on eyewitness testimony.

[2304] "Facial expression" refers to temporary emotions and reactions expressed through facial muscle movements, gaze, etc.

[2305] "Voice" refers to the linguistic expressions such as tone, intonation, and volume of the voice spoken by the witness.

[2306] "Emotional information" refers to data that represents the psychological state of a witness analyzed by the emotion engine.

[2307] "Interface" refers to the screen and operating means for exchanging information between the user and the system.

[2308] "Feedback" refers to the behavior of witnesses expressing their opinions or requests for corrections regarding the generated portrait.

[2309] "Server" refers to a remote computer for storing, processing, and transmitting data.

[2310] This invention is a system that acquires feature information based on eyewitness testimony and uses an emotion engine to generate and modify highly accurate caricatures in real time. The system takes into account the emotions of the eyewitness and modifies the caricature as many times as necessary until the eyewitness is satisfied, and finally saves it in a database.

[2311] System programs and the hardware and software used

[2312] The system consists of the following main components:

[2313] 1. Server: Generates and modifies caricatures based on feature and emotion information.

[2314] Software used: Emotion recognition software, caricature generation algorithm

[2315] Hardware used: high-speed processor, database server

[2316] 2. Terminal: An input device used by witnesses or investigators to transmit input feature information and feedback information to the server. Examples include smart glasses, smartphones, and tablets.

[2317] Software used: Input interface, emotion recognition software

[2318] Hardware used: Smart glasses (e.g., Google Glass, Vuzix Blade), smartphone (iOS / Android)

[2319] 3. User Interface (UI): A screen for witness and investigator interaction, displaying the sketch generated in real time and providing the ability to input witness feedback.

[2320] Software used: Browser-based UI, desktop application, mobile app

[2321] Explanation of program processing

[2322] The server uses emotion recognition software to recognize emotions from the witness's facial expressions and voice. This software analyzes the witness's facial expression data and voice data to recognize their current emotional state (e.g., nervous, anxious, relaxed). Based on the emotional information, the server dynamically adjusts the interface to provide an environment in which the witness can relax and easily testify.

[2323] When feature information is entered on the device, it is sent to the server, which then generates an initial caricature based on the feature information using a caricature generation algorithm. The generated caricature is then displayed to the witness through a user interface. When the witness provides feedback, the device sends the information to the server, which then modifies the caricature based on the feedback.

[2324] This process is repeated until the witness is satisfied, and once the final portrait is completed, the server stores it in a database.

[2325] Specific examples

[2326] For example, if a witness says, "The perpetrator has black hair, a round face, and narrow eyes," these characteristics are entered into the device. The server uses emotion recognition software to analyze whether the witness is nervous or relaxed. If it detects nervousness, the server adjusts the user interface, changing the interface design and the questions presented to put the witness at ease.

[2327] After the first sketch is generated and displayed on the device, the witness may point out that the eyes are narrower. This feedback is sent to the server, which then modifies the sketch based on the feedback. This process is repeated several times until a final sketch is created that satisfies the witness.

[2328] Prompt Sentence Examples

[2329] "Please tell us the characteristics of the perpetrator. Please be as detailed as possible. What is the shape of the eyes? What color is the hair? What is the shape of the face? Please relax and answer."

[2330] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2331] Step 1:

[2332] The server initializes the system. The terminal starts up and the investigator enters their login information. Once the login information is entered, the server verifies it and performs authentication. Once authentication is complete, the server establishes a connection to the database and initializes the portrait generation algorithm. At this point, the terminal displays a message indicating that initialization is complete.

[2333] Input: Investigator login information

[2334] Output: Authentication complete message, initial setup complete message

[2335] Data processing / calculation: Authenticating login information, establishing database connections, initializing algorithms

[2336] Step 2:

[2337] The user (investigator) asks the witness for basic information (e.g., age, gender, hair color, etc.) in an interview format. The basic information provided by the witness is entered into the device's UI. The device then transmits the entered characteristic information to the server in real time.

[2338] Input: basic information and characteristics of the witness

[2339] Output: Sending the input feature information

[2340] Data processing / calculation: Input and organization of basic information, real-time transmission of characteristic information

[2341] Step 3:

[2342] The server receives the feature information and uses an emotion engine to recognize emotions from the witness's facial expressions and voice. The recognized emotion information is analyzed on the server side, and the UI is dynamically adjusted to make the witness feel relaxed and easy to give feedback.

[2343] Input: witness's facial expression data, voice data

[2344] Output: Recognized emotion information

[2345] Data processing / calculation: Analysis of facial expression data and voice data, emotion recognition, dynamic UI adjustment

[2346] Step 4:

[2347] The server generates an initial portrait based on the received feature information. A portrait generation algorithm is used for this generation. The generated portrait data is sent to the terminal, and the initial portrait is displayed on the terminal.

[2348] Input: feature information

[2349] Output: Sending and displaying the initial portrait data

[2350] Data processing / calculation: Generate portraits based on feature information, send portrait data

[2351] Step 5:

[2352] The user (witness) checks the sketch and tells the investigator what they would like to be corrected (e.g., "Make the eyes thinner"). The investigator inputs the witness's feedback into the device's UI. The input correction requests are sent from the device to the server.

[2353] Input: Witness feedback, correction requests

[2354] Output: Sending correction requests

[2355] Data processing / calculation: Input and sorting of feedback, sending correction requests

[2356] Step 6:

[2357] The server then modifies the portrait based on the requested corrections. The modified portrait data is then sent back to the terminal and displayed on the device. This process is repeated until the witness is satisfied.

[2358] Input: desired corrections

[2359] Output: Send and display the corrected portrait data

[2360] Data processing / calculation: Correction of portraits based on desired correction points, resending of portrait data

[2361] Step 7:

[2362] The users (witness and investigator) review the final sketch and, if satisfied, agree on it as the final sketch. The investigator selects "Save," and the server saves the final sketch data in the database. A notification that saving is complete is displayed on the device.

[2363] Input: Final portrait data

[2364] Output: Database save, save completion notification

[2365] Data processing / calculation: Saving portrait data and sending a notification when saving is complete

[2366] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2367] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2368] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2369] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2370] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2371] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2372] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2373] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2374] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2375] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2376] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2377] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2378] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2379] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2380] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2381] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2382] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2383] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2384] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2385] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2386] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2387] The following is further disclosed regarding the above embodiment.

[2388] The following are proposed draft claims:

[2389] (Claim 1)

[2390] a means for inputting characteristic information from witnesses;

[2391] A means for generating a portrait based on input feature information;

[2392] a means for displaying the generated portrait to a witness;

[2393] a means for inputting feedback from witnesses;

[2394] a means for modifying the caricature based on the input feedback;

[2395] a means for displaying the modified portrait to the witness again;

[2396] The system includes means for storing the final portrait in a database.

[2397] (Claim 2)

[2398] means for transmitting the input characteristic information to a server;

[2399] The system of claim 1, further comprising: a server for generating a caricature based on the feature information.

[2400] (Claim 3)

[2401] means for transmitting the input feedback to a server;

[2402] and means for the server to modify the c...

Claims

1. a means for inputting characteristic information from witnesses; A means for generating a portrait based on input feature information; a means for displaying the generated portrait to a witness; a means for inputting feedback from witnesses; a means for modifying the caricature based on the input feedback; a means for displaying the modified portrait to the witness again; The system includes means for storing the final portrait in a database.

2. means for transmitting the input characteristic information to a server; The system of claim 1 , further comprising: a server for generating a caricature based on the characteristic information.

3. means for transmitting the input feedback to a server; and means for the server to modify the caricature based on the feedback.

Images