system

Abstract

We provide the system. [Solution] Means for collecting applicant data in various formats, A method for cross-analyzing data collected using generative AI, A means of evaluating applicants' creativity, technical skills, ethical values, and communication abilities, Based on these evaluations, a means of predicting the future performance and career growth of applicants, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern recruitment activities, it is difficult to comprehensively evaluate the diverse skills and characteristics of applicants and predict future performance and career growth based on them. In addition, there is a problem that suitable candidates for a company's long-term talent strategy cannot be sufficiently identified through selection that depends only on the past achievements and current abilities of applicants. Furthermore, due to the wide variety of data formats, there is also a problem that its integrated analysis is difficult.

Means for Solving the Problems

[0005] To solve the above problems, this invention provides a system that collects applicant data in various formats and clarifies applicant characteristics through cross-analysis using generating AI. Specifically, it includes means for evaluating creativity, technical skills, ethical values, and communication skills, and further predicts the applicant's future performance and career growth based on these evaluations. In this way, it improves the accuracy of talent selection in companies and supports the realization of sustainable talent strategies.

[0006] "Applicant data" refers to information provided by applicants, including past achievements and activity history, and encompasses a variety of formats such as paintings, essays, social media posts, and academic papers.

[0007] "Generative AI" refers to artificial intelligence that uses technologies such as machine learning and natural language processing to analyze data, extract patterns and features, and make predictions about the future.

[0008] "Cross-analysis" is a method for comprehensively analyzing data in various formats, identifying relationships between different data types, and conducting integrated evaluations.

[0009] "Creativity" refers to the ability to generate new ideas and concepts, and is a trait particularly sought after in the fields of technology and art.

[0010] "Technical ability" refers to the ability to solve problems by utilizing specialized knowledge and skills in a particular field.

[0011] "Ethical values" refer to the ability to make judgments and take actions based on social and moral standards, and include a sense of responsibility and social consideration.

[0012] "Communication skills" refer to the ability to effectively convey information to others and deepen mutual understanding.

[0013] "Future performance" refers to the results an applicant is expected to achieve within a specific future period, and includes projected performance and achievements.

[0014] "Career growth" refers to the improvement in job duties and positions, and the advancement of skills that applicants may experience throughout their professional lives. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system for comprehensively evaluating applicants and predicting their future performance in a company's recruitment process. This system is implemented through a series of processes involving a server, terminals, and users.

[0037] First, users use their devices to provide the system with various forms of data about themselves. This includes uploading data through online forms and sharing links to social media accounts. The server collects this data and retrieves external data, including social media posts, using appropriate APIs. This centralizes the collection of information related to applicants.

[0038] Next, the server prepares to process data of different formats in a unified manner for analysis. OCR technology is used for text extraction from image data, and natural language processing technology is used for analyzing text data. After these preparations are complete, the server performs cross-analysis of the data using generative AI to extract the characteristics and skills of applicants.

[0039] After analysis, the server evaluates the applicant's creativity, technical skills, ethics, and communication abilities. AI scoring is applied to each evaluation item, and an applicant profile is generated based on these scores. This profile is displayed on a dashboard accessible to recruiters via their devices. The dashboard visualizes the evaluation results and strengths and weaknesses, allowing users to quickly understand the applicant's characteristics.

[0040] Furthermore, the server uses a model to predict future performance and plots and visualizes the career growth of applicants. This allows companies to make selections that take long-term talent strategies into consideration. For example, if a user applies for a design position, their design sense and speed of skill acquisition are evaluated and predicted based on their past work and submissions, and their future potential is presented. Through this series of processes, the present invention helps companies clearly understand the overall picture of applicants and select the most suitable talent.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] Users enter and submit application information using their devices. This includes uploading various files, such as resume data, links to social media accounts, and digital copies of past papers and works.

[0044] Step 2:

[0045] The server uses links provided by the user to retrieve relevant data from social media and other external resources via APIs. This completes the collection of all data related to the applicant.

[0046] Step 3:

[0047] The server formats the data. Specifically, it uses OCR to extract text from images and converts different data formats (e.g., text, images, documents) into a format that can be parsed.

[0048] Step 4:

[0049] The server runs a generative AI model to analyze the formatted data. Here, natural language processing is used to analyze textual data, and image analysis techniques are used to evaluate visual data.

[0050] Step 5:

[0051] Based on the analysis results, the server scores applicants' creativity, technical skills, ethical values, and communication abilities. This data is then used to generate an evaluation profile of the applicant.

[0052] Step 6:

[0053] On their devices, recruiters can visually review applicant evaluation results via a dashboard. The dashboard graphs the scored results, clearly displaying strengths and weaknesses.

[0054] Step 7:

[0055] The server uses a predictive model to forecast applicants' future performance and career growth. This information is also displayed on the dashboard.

[0056] Step 8:

[0057] Users and recruiters will use the information gathered to select the most suitable candidates and make decisions based on their long-term talent strategy.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] In traditional talent selection processes, it is difficult to effectively utilize diverse forms of applicant data and comprehensively evaluate candidates based on criteria such as creativity, technical skills, and ethical values. Furthermore, there is a lack of means to predict future performance and understand career growth. This makes it difficult for companies to select appropriate talent.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for collecting and centrally managing data in various formats from applicants, means for pre-processing information to extract information from images and text, and means for analyzing the collected information using a generative model to extract the characteristics of the applicants. This enables a multifaceted analysis of applicant information, allowing for evaluation based on creativity, technical ability, ethical values, and communication skills, as well as prediction of future job suitability and career growth.

[0063] An "applicant" is an individual who expresses interest in a job or role offered by a company or organization and submits information about it.

[0064] "Data in diverse formats" refers to all information provided in different formats and media, such as text, images, social media posts, and resume information.

[0065] "Centralized management" refers to the process of integrating data from different formats and sources to make it easier to handle within a single system.

[0066] "Preprocessing" refers to initial information processing activities to convert data into an analyzable format, and specifically includes text extraction from images and format conversion of text data.

[0067] A "generative model" refers to an algorithm or structure that uses artificial intelligence technology to learn features from large amounts of data and perform analysis and prediction of new information.

[0068] "Applicant characteristics" refer to distinctive data and information regarding the skills, abilities, values, and other characteristics of each individual applicant.

[0069] "Creativity" refers to the ability to create new value based on existing information and resources.

[0070] "Technical evaluation" refers to the activity of assessing the specialized knowledge and technical skills that applicants possess.

[0071] "Ethical values" refer to the moral judgment and values ​​that an applicant demonstrates, meaning they possess the right standards for judgment and action.

[0072] "Communication skills" refer to the ability to effectively exchange information with others and clearly convey one's intentions.

[0073] "Job suitability" refers to characteristics that indicate how adaptable and talented an individual is to a particular job or task.

[0074] "Job growth" refers to the predicted future scenario in which an applicant is likely to develop and grow in their career.

[0075] This system is designed to comprehensively evaluate applicants in a company's recruitment process and predict their future performance. The system is implemented through a process involving servers, terminals, and users.

[0076] Users use their devices to enter the data required for their application. This data includes information entered into online forms and the submission of links to their social media accounts. The server collects data from users and uses appropriate APIs to import external data, including social media posts, to centrally aggregate information related to applicants.

[0077] The server preprocesses the collected data. For image data, it extracts text using OCR technology, and applies natural language processing techniques to the text data. To perform this processing, it utilizes tools and frameworks such as TENSORFLOW® and OpenCV.

[0078] The server uses a generative AI model to perform cross-analysis on the aggregated data. This analysis can identify characteristics such as applicants' creativity, technical skills, ethical values, and communication abilities. Based on the evaluated data, predictions are made about their future job suitability and career growth.

[0079] The analysis results and predictions are visualized on a dashboard generated on the server, which recruiters can access via their devices. The dashboard displays the evaluated results in graphs and charts, providing the information necessary for hiring decisions.

[0080] For example, when a user applies for a design position, the server analyzes their past design work and relevant social media posts to evaluate and predict their design sense and learning speed. Through this process, companies can gain a clear overall understanding of the applicant and receive support in selecting the right talent.

[0081] An example of a prompt is: "Evaluate the applicant's creativity and learning speed in a design role, and predict their future performance."

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] Users use their devices to enter the necessary data for their application into an online form. This data includes contact information, work experience, educational background, and social media account information. This information is entered into the system for the first time and is sent to the server.

[0085] Step 2:

[0086] The server initiates a process to collect and centrally manage data received from users. Utilizing SNS APIs, it retrieves relevant posting data from provided accounts and adds it to the system as already acquired information. This process integrates information from multiple data sources to form a complete applicant profile.

[0087] Step 3:

[0088] The server uses OCR and natural language processing technologies to preprocess the collected data. The input for this step is image and text data; OCR extracts text from images. Natural language processing tokenizes the text data and identifies important words such as nouns and verbs, preparing it for subsequent analysis.

[0089] Step 4:

[0090] After preprocessing, the server performs analysis using a generative AI model. This step reveals the characteristics of the applicants, such as creativity and technical skills. The input to this analysis is the output data from step 3, and the output is a profile of each applicant's characteristics. The analysis process applies machine learning algorithms to find correlations between the data.

[0091] Step 5:

[0092] The server predicts the applicant's future job suitability and career growth based on the analysis results. Using the trait profile obtained in Step 4 as input, a machine learning model simulates the future career path. The output of this step is a report showing the applicant's predicted growth curve and suitability.

[0093] Step 6:

[0094] The server generates a dashboard to visually display the obtained analysis results and predictions. In this step, the output data from step 5 is used as input and visualized in graph and chart format. The output is a visualized dashboard that the user can view through their terminal.

[0095] Step 7:

[0096] Users access the dashboard from their devices to view all information about applicants. Based on this information, users can gain data-driven insights to support decision-making regarding recruitment and talent strategy.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In corporate recruitment activities, there is a need to quickly and accurately evaluate applicants' abilities and characteristics, and to support long-term decisions that enhance their future job performance and suitability for the organization. Furthermore, security risk assessment is also crucial, and there is a need for a system that can comprehensively perform these tasks.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes a device for aggregating applicant information in various formats, a device for analyzing the aggregated information using a generating AI, and a device for determining security clearance based on the applicant data. This enables a multifaceted evaluation of applicant characteristics and comprehensive support for talent selection, including security risks.

[0102] "Applicant information in various formats" refers to information in various data formats provided by applicants, including text data, image data, and online profile information.

[0103] "Aggregation device" refers to hardware or software for centrally collecting and integrating applicant information in different formats.

[0104] "Generative AI" refers to artificial intelligence technology used to extract and analyze the characteristics of applicants based on given data.

[0105] "Analysis device" refers to hardware or software that performs the necessary computational processing to analyze the collected applicant information.

[0106] "Creativity, technical skills, ethics, and communication skills" are indicators used to evaluate the characteristics of applicants, and include the novelty of deliverables, technical skills, moral judgment, and interpersonal communication skills.

[0107] A "device for predicting future abilities and career growth" refers to hardware or software that predicts an applicant's future work performance and career advancement based on their past data and characteristics.

[0108] "Security clearance" refers to an indicator that determines the level of information an applicant is permitted to access within an organization, based on background checks and character assessments.

[0109] A "display platform" refers to an interface that visually displays analysis results, allowing users to easily understand the characteristics of applicants.

[0110] "A device that supports the selection of organizational members" refers to hardware or software that selects the most suitable personnel based on analysis and prediction results, and supports recruitment activities.

[0111] The system that implements this application consists of a server, terminals, and users. The server plays the main role in information processing, aggregating applicant information in various formats and analyzing that data. Specifically, the server collects information from text data, image data, and online profiles obtained from applicants, extracts text from images using Tesseract OCR, and performs natural language processing on the text data using the spaCy library. Through this series of data processing, the characteristics and skills of applicants are extracted. After further analysis of the extracted data using generative AI technology, the applicant's future abilities and career growth are predicted, and the results are stored in the cloud and used as indicators to support corporate decision-making.

[0112] The terminal functions as an interface for providing applicant evaluation results to HR personnel. The evaluation results are visualized via a display platform, allowing personnel to refer to them and understand the applicant's characteristics. Users can access this service via smartphones or computers, enter applicant data, and view the resulting generated profiles and security clearance information.

[0113] As a concrete example, when a company hires a new security officer, there is a process to quickly determine the security clearance level based on the applicant's social media data and skill information. The prompt input to the generating AI model is in the format of, "Please perform an appropriate security assessment based on the data provided by this applicant," which prompts the AI ​​to start the analysis.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The server receives text and image data of applicants submitted by users. It takes data uploaded via online forms and social media links as input and organizes it into a format for centralized storage in a database on the server as output. This data is then converted to an appropriate format for subsequent analysis.

[0117] Step 2:

[0118] The server extracts text from image data using Tesseract OCR. The input is the image data collected in step 1, and the output is the extracted text data. In this data conversion process, the character information contained in the image is converted into text and sent to the next step.

[0119] Step 3:

[0120] The server uses the spaCy library to analyze text data. The input is the text data obtained in the previous step, and the output is information about the applicant's characteristics and traits. Through natural language processing, meaningful information is extracted from the text, and the applicant's characteristics are identified.

[0121] Step 4:

[0122] The server cross-analyzes the data analyzed using a generative AI model. Here, the characteristics of each applicant are input into the model. The generative AI model starts processing with the prompt message "Please perform an appropriate security assessment based on the data provided by this applicant," and outputs a comprehensive report including the applicant's security clearance.

[0123] Step 5:

[0124] The terminal displays the analysis results sent from the server. The input is the analysis report from the server, and the output is a dashboard that the user can visually review. The terminal graphically represents the evaluation results of characteristics and clearance levels, allowing the person in charge to easily evaluate applicants.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention is a system for a company's recruitment activities that comprehensively evaluates applicant data and predicts future performance. Furthermore, by incorporating an emotion engine that recognizes user emotions and utilizes that data for analysis, the accuracy of the evaluation is improved.

[0127] First, users provide the system with various forms of data about themselves using their devices. This data includes resumes, social media posts, and past works. The server collects this data and, if necessary, uses APIs to retrieve additional information from external data sources.

[0128] Next, the server preprocesses the data. It unifies the data format by extracting text from images using OCR technology and analyzing the sentences using natural language processing. During this process, the sentiment engine analyzes the user's emotions from the collected texts and social media posts, identifying positive, negative, and neutral emotions.

[0129] Next, a cross-analysis using generative AI is performed to evaluate the applicant's creativity, technical skills, ethics, and communication abilities. Emotional data obtained by the emotion engine is used to enhance the evaluation of communication abilities in particular. For example, the tone of the applicant's statements on social media is analyzed to provide additional insights into their interpersonal skills.

[0130] The server generates an evaluation profile of the applicant based on each analyzed evaluation element. This evaluation profile is displayed on a dashboard that recruiters access via their terminals. The dashboard visualizes the scored items, clearly showing the applicant's characteristics.

[0131] Furthermore, the server uses a predictive model to calculate the applicant's future performance and career growth potential. This predictive information is also reflected in the dashboard, which companies can use as a reference to build long-term talent strategies. For example, for an applicant aspiring to a design position, their design sense is evaluated based on their past work, and their past challenges and feelings of praise are analyzed through an emotion engine to predict their future potential.

[0132] In this way, this system evaluates applicants from multiple perspectives, helping companies efficiently select the most suitable personnel.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] Users submit their application data to the system using their devices. This process includes uploading resumes and past work files, and providing links to their social media accounts.

[0136] Step 2:

[0137] The server collects data from users and, if necessary, uses APIs to retrieve publicly available information from social networking platforms. This allows for the acquisition of additional data from users' online activities.

[0138] Step 3:

[0139] The server formats the collected data into a processable format. Specifically, it applies OCR technology to extract text from images and performs natural language processing on document data to analyze its content.

[0140] Step 4:

[0141] The emotion engine is activated to analyze the emotions contained in the user's text data and social media posts. This identifies and records positive, negative, or neutral emotions from the user's posts.

[0142] Step 5:

[0143] The server uses a generative AI model to perform analysis and evaluate applicants' creativity, technical skills, ethics, and communication abilities. Emotional data obtained from an emotion engine is used to add depth to the evaluation, particularly of communication skills.

[0144] Step 6:

[0145] The server generates applicant profiles based on the evaluation results. The evaluated scores are incorporated into a dashboard and displayed visually. Users and company recruiters can access this information to review the evaluation details.

[0146] Step 7:

[0147] The server uses a predictive model to forecast applicants' future performance and career growth. This forecast includes collected data and sentiment data, which are visualized on a dashboard.

[0148] Step 8:

[0149] Recruiters use a dashboard on their devices to make decisions about selecting the best candidates, based on an overall evaluation of applicants and their future projections. This process allows companies to implement their talent strategies more effectively.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] In modern recruitment practices, applicant evaluations are often based solely on superficial data, making it difficult to accurately assess an individual's true abilities and potential. In particular, the lack of evaluation that considers emotional insights and potential results in companies being unable to efficiently select the best talent.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes means for collecting personal information in various forms, means for extracting information from images using optical character recognition technology, means for analyzing the extracted text using natural language processing technology, means for analyzing an individual's emotional state using sentiment analysis technology, means for performing cross-analysis of the collected information using a generative AI model, means for evaluating an individual's creativity, technical skills, ethical values, and communication abilities, and means for predicting an individual's future performance and career growth potential based on these evaluations. This enables a comprehensive evaluation that takes into account not only superficial information but also an individual's potential and emotional insights.

[0155] "Various forms of personal information" refers to various data formats and types related to an individual, including information such as resumes, social media posts, image files, and document files.

[0156] "Optical character recognition technology" is a technology for extracting text data contained in images, handwritten documents, etc., as digital characters.

[0157] "Natural language processing technology" is a technology that analyzes the language that humans speak and write naturally and understands its meaning.

[0158] "Emotion analysis technology" is a technique that identifies emotions from text data and specifies emotional states such as positive, negative, or neutral.

[0159] A "generative AI model" is a group of algorithms that use artificial intelligence to generate and analyze data, and is used to analyze multiple data sources and evaluation factors in a unified manner.

[0160] Cross-analysis is a technique that analyzes interrelationships and trends by combining different datasets or evaluation factors.

[0161] An "evaluation profile" is a report that quantifies and visualizes an individual's various abilities and characteristics, and includes the results of their evaluation.

[0162] To implement this invention, a server, a terminal, and a user must work together. First, the user uses a terminal to provide the system with various forms of personal information, such as resumes, social media posts, and past works. The terminal converts this data into an appropriate format and sends it to the server.

[0163] The server uses the following technologies to process this personal information: it extracts text from image files using OCR technology, and analyzes the grammar and keywords of the extracted text using natural language processing (NLP) technology. It also analyzes the emotional state derived from the text using sentiment analysis technology to identify the individual's emotional state.

[0164] Next, the server uses a generative AI model to cross-analyze the collected personal information. This analysis evaluates an individual's creativity, technical skills, ethical values, and communication abilities, and generates an evaluation profile. Based on this profile, it predicts the individual's future performance and career growth potential.

[0165] For example, if an applicant is applying for a design position, they would upload a resume that includes data on how their past design work was created. The server would then use OCR technology to convert the resume image into text and NLP technology to analyze design-related skills and context. Furthermore, it would perform sentiment analysis on the applicant's social media posts to analyze their emotional state.

[0166] An example of a prompt using a generative AI model is "Please provide insights into the applicant's creative activities," which is then entered to start the analysis process. In this way, companies can obtain a comprehensive evaluation of applicants and efficiently select the most suitable talent.

[0167] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0168] Step 1:

[0169] Users input personal information such as resumes, social media posts, and past works into the system via their terminal. The input data consists of different formats (e.g., PDF, JPEG, text files). The terminal converts this data into a specific format and prepares it for transmission to the server.

[0170] Step 2:

[0171] The server receives personal information transmitted from the terminal. It then uses optical character recognition (OCR) to extract text data from image files. For example, it extracts text information from a scanned resume image to prepare for the next processing step.

[0172] Step 3:

[0173] The server utilizes natural language processing (NLP) technology to analyze the extracted text data. The input is text data from OCR, and it performs grammatical structure analysis and keyword extraction, determining the meaning based on that analysis. This forms the foundation for a deep understanding of user resumes and social media posts.

[0174] Step 4:

[0175] The server applies sentiment analysis techniques to identify the user's emotional state from text analyzed by NLP. For example, positive, negative, or neutral emotional states are identified within the context of a social media post. The resulting sentiment data is then used in subsequent cross-analysis.

[0176] Step 5:

[0177] The server uses a generative AI model to perform a complex cross-analysis using NLP analysis results and sentiment data as input. This analysis evaluates the user's creativity, technical skills, ethics, and communication abilities. An example of a prompt for the AI ​​model is, "Please provide insights into the applicant's creative activities."

[0178] Step 6:

[0179] The server generates an evaluation profile of the applicant based on the results of the cross-analysis. At this stage, each evaluation item is scored, and the user's characteristics are quantified. The generated profile is used in the next step.

[0180] Step 7:

[0181] Through the terminal, recruiters access a user interface that visualizes the generated evaluation profile. Graphs and charts displayed on the dashboard allow for a visual assessment of the user's abilities and potential. This provides companies with information to support their hiring decisions.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] In modern manufacturing environments, numerous machines operate in parallel, making their optimal placement and performance evaluation crucial. However, there is a lack of systems for objectively and comprehensively evaluating machine operation and performance. In particular, the inability to obtain intuitive understanding through emotional evaluations leads to decreased operational efficiency.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes means for collecting evaluation data in various formats, means for cross-analyzing the collected data using a generating AI, and means for emotionally evaluating the operating state using an emotion analysis engine. This makes it possible to improve the suitability evaluation and operational efficiency of machines in manufacturing sites.

[0187] "Evaluation target" is a general term for the objects analyzed by the system, and in this case, it refers to machines and robots operating within a factory.

[0188] "Generative AI" is a type of artificial intelligence technology that uses algorithms to analyze large amounts of data and derive relationships between them.

[0189] "Data collection means" refers to the technology or device used to collect necessary information from the subject of evaluation.

[0190] Cross-analysis is an analytical method that uses different perspectives and criteria to perform a multifaceted evaluation of collected data.

[0191] An "emotion analysis engine" is an analytical device or software that derives emotional meaning from the content and behavior of data.

[0192] The "integrated display surface" is an interface for displaying analysis results in an intuitively understandable format, and is provided as graphs and dashboards.

[0193] This system is designed to evaluate the suitability of machinery in a manufacturing environment and to enable efficient operation. The system is primarily composed of three components: a server, terminals, and users.

[0194] The server collects evaluation data in various formats and cross-analyzes it using generative AI. Specifically, the server extracts necessary text data from images using OCR technology and consolidates various operational data acquired from sensors into a unified format. Based on the collected data, the generative AI performs analysis to comprehensively evaluate the creativity, technical capabilities, and efficiency of machines and robots.

[0195] The emotion analysis engine analyzes video data and motion data, evaluating, for example, smooth movements as "positive emotions" and unnecessary movements or obstacles as "negative emotions." This analysis is performed using natural language processing such as IBM Watson® Natural Language Understanding. The evaluation results are visualized on a dashboard, which is an integrated display surface, and provided to the user via the terminal.

[0196] The terminals used include smartphones and head-mounted displays, allowing for visual confirmation of data and viewing of analysis results. This information enables users to make appropriate decisions based on machine data in the manufacturing environment.

[0197] For example, consider a factory robot A that efficiently streamlines line work with its rapid movements and optimizes power consumption. This evaluation is reflected as "high efficiency" on the dashboard. Conversely, if machine B frequently stops working, it will be evaluated as "low efficiency," and maintenance will be recommended.

[0198] An example of a prompt might be text such as, "Analyze the robot's motion and emotionally evaluate its efficiency and performance." This allows the manufacturing environment to achieve higher productivity and efficiency.

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The server collects operational data from each machine in the factory. Through sensors and IoT devices, it acquires data such as operating time, failure frequency, and power consumption for each machine. This data is stored in a database and formatted into a unified format. The input is raw data from each machine, and the output is a formatted dataset.

[0202] Step 2:

[0203] The server uses OCR technology on the formatted dataset to extract necessary text information from the image data. Here, Tesseract OCR is used to obtain metrics from images of the machine's dashboard and main screen. The input is an image file, and the output is text data.

[0204] Step 3:

[0205] The server utilizes generative AI to perform cross-analysis based on the collected data. This analysis scores the creativity, technical capabilities, and efficiency of machines and robots. The input is formatted and OCR-processed data, and the output is a score list representing the analysis results.

[0206] Step 4:

[0207] The server uses an emotion analysis engine to emotionally evaluate the operating state of each machine. Utilizing IBM Watson Natural Language Understanding, it evaluates smoothness and efficiency of operation as "positive emotions" and analyzes malfunctions and shutdowns as "negative emotions." The input is machine operation data, and the output is an emotion evaluation score.

[0208] Step 5:

[0209] The terminal visualizes the analysis results received from the server and displays them on a dashboard. Users can then use this to intuitively understand the machine's operating status and efficiency. The input is scored evaluation data, and the output is a visualized dashboard.

[0210] Step 6:

[0211] Users can review machine operation and layout on the factory floor based on visualized information. They can also determine the need for maintenance based on the analysis results. The input is evaluation information displayed on the dashboard, and the output is the user's decisions and actions.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] This invention is a system for comprehensively evaluating applicants and predicting their future performance in a company's recruitment process. This system is implemented through a series of processes involving a server, terminals, and users.

[0229] First, users use their devices to provide the system with various forms of data about themselves. This includes uploading data through online forms and sharing links to social media accounts. The server collects this data and retrieves external data, including social media posts, using appropriate APIs. This centralizes the collection of information related to applicants.

[0230] Next, the server prepares to process data of different formats in a unified manner for analysis. OCR technology is used for text extraction from image data, and natural language processing technology is used for analyzing text data. After these preparations are complete, the server performs cross-analysis of the data using generative AI to extract the characteristics and skills of applicants.

[0231] After analysis, the server evaluates the applicant's creativity, technical skills, ethics, and communication abilities. AI scoring is applied to each evaluation item, and an applicant profile is generated based on these scores. This profile is displayed on a dashboard accessible to recruiters via their devices. The dashboard visualizes the evaluation results and strengths and weaknesses, allowing users to quickly understand the applicant's characteristics.

[0232] Furthermore, the server uses a model to predict future performance and plots and visualizes the career growth of applicants. This allows companies to make selections that take long-term talent strategies into consideration. For example, if a user applies for a design position, their design sense and speed of skill acquisition are evaluated and predicted based on their past work and submissions, and their future potential is presented. Through this series of processes, the present invention helps companies clearly understand the overall picture of applicants and select the most suitable talent.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] Users enter and submit application information using their devices. This includes uploading various files, such as resume data, links to social media accounts, and digital copies of past papers and works.

[0236] Step 2:

[0237] The server uses links provided by the user to retrieve relevant data from social media and other external resources via APIs. This completes the collection of all data related to the applicant.

[0238] Step 3:

[0239] The server formats the data. Specifically, it uses OCR to extract text from images and converts different data formats (e.g., text, images, documents) into a format that can be parsed.

[0240] Step 4:

[0241] The server runs a generative AI model to analyze the formatted data. Here, natural language processing is used to analyze textual data, and image analysis techniques are used to evaluate visual data.

[0242] Step 5:

[0243] Based on the analysis results, the server scores applicants' creativity, technical skills, ethical values, and communication abilities. This data is then used to generate an evaluation profile of the applicant.

[0244] Step 6:

[0245] On their devices, recruiters can visually review applicant evaluation results via a dashboard. The dashboard graphs the scored results, clearly displaying strengths and weaknesses.

[0246] Step 7:

[0247] The server uses a predictive model to forecast applicants' future performance and career growth. This information is also displayed on the dashboard.

[0248] Step 8:

[0249] Users and recruiters will use the information gathered to select the most suitable candidates and make decisions based on their long-term talent strategy.

[0250] (Example 1)

[0251] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0252] In traditional talent selection processes, it is difficult to effectively utilize diverse forms of applicant data and comprehensively evaluate candidates based on criteria such as creativity, technical skills, and ethical values. Furthermore, there is a lack of means to predict future performance and understand career growth. This makes it difficult for companies to select appropriate talent.

[0253] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0254] In this invention, the server includes means for collecting and centrally managing data in various formats from applicants, means for pre-processing information to extract information from images and text, and means for analyzing the collected information using a generative model to extract the characteristics of the applicants. This enables a multifaceted analysis of applicant information, allowing for evaluation based on creativity, technical ability, ethical values, and communication skills, as well as prediction of future job suitability and career growth.

[0255] An "applicant" is an individual who expresses interest in a job or role offered by a company or organization and submits information about it.

[0256] "Data in diverse formats" refers to all information provided in different formats and media, such as text, images, social media posts, and resume information.

[0257] "Centralized management" refers to the process of integrating data from different formats and sources to make it easier to handle within a single system.

[0258] "Preprocessing" refers to initial information processing activities to convert data into an analyzable format, and specifically includes text extraction from images and format conversion of text data.

[0259] A "generative model" refers to an algorithm or structure that uses artificial intelligence technology to learn features from large amounts of data and perform analysis and prediction of new information.

[0260] "Applicant characteristics" refer to distinctive data and information regarding the skills, abilities, values, and other characteristics of each individual applicant.

[0261] "Creativity" refers to the ability to create new value based on existing information and resources.

[0262] "Technical evaluation" refers to the activity of assessing the specialized knowledge and technical skills that applicants possess.

[0263] "Ethical values" refer to the moral judgment and values ​​that an applicant demonstrates, meaning they possess the right standards for judgment and action.

[0264] "Communication skills" refer to the ability to effectively exchange information with others and clearly convey one's intentions.

[0265] "Job suitability" refers to characteristics that indicate how adaptable and talented an individual is to a particular job or task.

[0266] "Job growth" refers to the predicted future scenario in which an applicant is likely to develop and grow in their career.

[0267] This system is designed to comprehensively evaluate applicants in a company's recruitment process and predict their future performance. The system is implemented through a process involving servers, terminals, and users.

[0268] Users use their devices to enter the data required for their application. This data includes information entered into online forms and the submission of links to their social media accounts. The server collects data from users and uses appropriate APIs to import external data, including social media posts, to centrally aggregate information related to applicants.

[0269] The server preprocesses the collected data. For image data, it uses OCR technology to extract text, and applies natural language processing techniques to the text data. Tools and frameworks such as TensorFlow and OpenCV are used to perform this processing.

[0270] The server uses a generative AI model to perform cross-analysis on the aggregated data. This analysis can identify characteristics such as applicants' creativity, technical skills, ethical values, and communication abilities. Based on the evaluated data, predictions are made about their future job suitability and career growth.

[0271] The analysis results and predictions are visualized on a dashboard generated on the server, which recruiters can access via their devices. The dashboard displays the evaluated results in graphs and charts, providing the information necessary for hiring decisions.

[0272] For example, when a user applies for a design position, the server analyzes their past design work and relevant social media posts to evaluate and predict their design sense and learning speed. Through this process, companies can gain a clear overall understanding of the applicant and receive support in selecting the right talent.

[0273] An example of a prompt is: "Evaluate the applicant's creativity and learning speed in a design role, and predict their future performance."

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] Users use their devices to enter the necessary data for their application into an online form. This data includes contact information, work experience, educational background, and social media account information. This information is entered into the system for the first time and is sent to the server.

[0277] Step 2:

[0278] The server initiates a process to collect and centrally manage data received from users. Utilizing SNS APIs, it retrieves relevant posting data from provided accounts and adds it to the system as already acquired information. This process integrates information from multiple data sources to form a complete applicant profile.

[0279] Step 3:

[0280] The server uses OCR and natural language processing technologies to preprocess the collected data. The input for this step is image and text data; OCR extracts text from images. Natural language processing tokenizes the text data and identifies important words such as nouns and verbs, preparing it for subsequent analysis.

[0281] Step 4:

[0282] After preprocessing, the server performs analysis using a generative AI model. This step reveals the characteristics of the applicants, such as creativity and technical skills. The input to this analysis is the output data from step 3, and the output is a profile of each applicant's characteristics. The analysis process applies machine learning algorithms to find correlations between the data.

[0283] Step 5:

[0284] The server predicts the future job suitability and career growth of the applicant based on the analysis results. Using the characteristic profile obtained in Step 4 as input, it simulates the future career path with a machine learning model. The output of this step is a report indicating the predicted growth curve and suitability of the applicant.

[0285] Step 6:

[0286] The server generates a dashboard to visually display the obtained analysis results and predictions. In this step, the output data of Step 5 is used as input and visualized in the form of graphs and charts. The output is a visualized dashboard that can be browsed by the user through the terminal.

[0287] Step 7:

[0288] The user accesses the dashboard from the terminal and checks all the information about the applicant. Based on this information, the user can obtain data-driven insights to assist in decision-making regarding recruitment and talent strategies.

[0289] (Application Example 1)

[0290] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0291] In the recruitment activities of an enterprise, it is required to quickly and accurately evaluate the abilities and characteristics of applicants and support long-term judgments to enhance future job performance and organizational fit. Also, risk assessment in terms of security is important, and there is a need for a system that can comprehensively perform these.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0293] In this invention, the server includes a device for aggregating applicant information in various formats, a device for analyzing the aggregated information using a generating AI, and a device for determining security clearance based on the applicant data. This enables a multifaceted evaluation of applicant characteristics and comprehensive support for talent selection, including security risks.

[0294] "Applicant information in various formats" refers to information in various data formats provided by applicants, including text data, image data, and online profile information.

[0295] "Aggregation device" refers to hardware or software for centrally collecting and integrating applicant information in different formats.

[0296] "Generative AI" refers to artificial intelligence technology used to extract and analyze the characteristics of applicants based on given data.

[0297] "Analysis device" refers to hardware or software that performs the necessary computational processing to analyze the collected applicant information.

[0298] "Creativity, technical skills, ethics, and communication skills" are indicators used to evaluate the characteristics of applicants, and include the novelty of deliverables, technical skills, moral judgment, and interpersonal communication skills.

[0299] A "device for predicting future abilities and career growth" refers to hardware or software that predicts an applicant's future work performance and career advancement based on their past data and characteristics.

[0300] "Security clearance" refers to an indicator that determines the level of information an applicant is permitted to access within an organization, based on background checks and character assessments.

[0301] A "display platform" refers to an interface that visually displays analysis results, allowing users to easily understand the characteristics of applicants.

[0302] "A device that supports the selection of organizational members" refers to hardware or software that selects the most suitable personnel based on analysis and prediction results, and supports recruitment activities.

[0303] The system that implements this application consists of a server, terminals, and users. The server plays the main role in information processing, aggregating applicant information in various formats and analyzing that data. Specifically, the server collects information from text data, image data, and online profiles obtained from applicants, extracts text from images using Tesseract OCR, and performs natural language processing on the text data using the spaCy library. Through this series of data processing, the characteristics and skills of applicants are extracted. After further analysis of the extracted data using generative AI technology, the applicant's future abilities and career growth are predicted, and the results are stored in the cloud and used as indicators to support corporate decision-making.

[0304] The terminal functions as an interface for providing applicant evaluation results to HR personnel. The evaluation results are visualized via a display platform, allowing personnel to refer to them and understand the applicant's characteristics. Users can access this service via smartphones or computers, enter applicant data, and view the resulting generated profiles and security clearance information.

[0305] As a concrete example, when a company hires a new security officer, there is a process to quickly determine the security clearance level based on the applicant's social media data and skill information. The prompt input to the generating AI model is in the format of, "Please perform an appropriate security assessment based on the data provided by this applicant," which prompts the AI ​​to start the analysis.

[0306] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0307] Step 1:

[0308] The server receives the applicant's text data and image data sent from the user. It has data uploaded from an online form or SNS link as input and arranges it in a format that is centrally stored in the database within the server as output. Since this data is used in subsequent analysis, it is converted into an appropriate format.

[0309] Step 2:

[0310] The server extracts text from the image data using Tesseract OCR. The input is the image-formatted data collected in Step 1, and the output is the extracted text data. In this data conversion process, the character information contained in the image is converted into text and sent to the next step.

[0311] Step 3:

[0312] The server analyzes the text data using the spaCy library. The input is the text data obtained in the previous step, and the output is information regarding the characteristics and traits of the applicant. Through natural language processing, meaningful information is extracted from the text, and the characteristics of the applicant are identified.

[0313] Step 4:

[0314] The server cross-analyzes the analyzed data using a generative AI model. Here, the characteristics regarding each applicant are input into the model. The generative AI model starts the process using the prompt sentence "Based on the provided data of this applicant, please conduct an appropriate security evaluation." and outputs a comprehensive report including the applicant's security clearance.

[0315] Step 5:

[0316] The terminal displays the analysis results sent from the server. The input is the analysis report from the server, and the output is a dashboard that the user can visually review. The terminal graphically represents the evaluation results of characteristics and clearance levels, allowing the person in charge to easily evaluate applicants.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] This invention is a system for a company's recruitment activities that comprehensively evaluates applicant data and predicts future performance. Furthermore, by incorporating an emotion engine that recognizes user emotions and utilizes that data for analysis, the accuracy of the evaluation is improved.

[0319] First, users provide the system with various forms of data about themselves using their devices. This data includes resumes, social media posts, and past works. The server collects this data and, if necessary, uses APIs to retrieve additional information from external data sources.

[0320] Next, the server preprocesses the data. It unifies the data format by extracting text from images using OCR technology and analyzing the sentences using natural language processing. During this process, the sentiment engine analyzes the user's emotions from the collected texts and social media posts, identifying positive, negative, and neutral emotions.

[0321] Next, a cross-analysis using generative AI is performed to evaluate the applicant's creativity, technical skills, ethics, and communication abilities. Emotional data obtained by the emotion engine is used to enhance the evaluation of communication abilities in particular. For example, the tone of the applicant's statements on social media is analyzed to provide additional insights into their interpersonal skills.

[0322] The server generates an evaluation profile of the applicant based on each analyzed evaluation element. This evaluation profile is displayed on a dashboard that recruiters access via their terminals. The dashboard visualizes the scored items, clearly showing the applicant's characteristics.

[0323] Furthermore, the server uses a predictive model to calculate the applicant's future performance and career growth potential. This predictive information is also reflected in the dashboard, which companies can use as a reference to build long-term talent strategies. For example, for an applicant aspiring to a design position, their design sense is evaluated based on their past work, and their past challenges and feelings of praise are analyzed through an emotion engine to predict their future potential.

[0324] In this way, this system evaluates applicants from multiple perspectives, helping companies efficiently select the most suitable personnel.

[0325] The following describes the processing flow.

[0326] Step 1:

[0327] Users submit their application data to the system using their devices. This process includes uploading resumes and past work files, and providing links to their social media accounts.

[0328] Step 2:

[0329] The server collects data from users and, if necessary, uses APIs to retrieve publicly available information from social networking platforms. This allows for the acquisition of additional data from users' online activities.

[0330] Step 3:

[0331] The server formats the collected data into a processable format. Specifically, it applies OCR technology to extract text from images and performs natural language processing on document data to analyze its content.

[0332] Step 4:

[0333] The emotion engine is activated to analyze the emotions contained in the user's text data and social media posts. This identifies and records positive, negative, or neutral emotions from the user's posts.

[0334] Step 5:

[0335] The server uses a generative AI model to perform analysis and evaluate applicants' creativity, technical skills, ethics, and communication abilities. Emotional data obtained from an emotion engine is used to add depth to the evaluation, particularly of communication skills.

[0336] Step 6:

[0337] The server generates applicant profiles based on the evaluation results. The evaluated scores are incorporated into a dashboard and displayed visually. Users and company recruiters can access this information to review the evaluation details.

[0338] Step 7:

[0339] The server uses a predictive model to forecast applicants' future performance and career growth. This forecast includes collected data and sentiment data, which are visualized on a dashboard.

[0340] Step 8:

[0341] Recruiters use a dashboard on their devices to make decisions about selecting the best candidates, based on an overall evaluation of applicants and their future projections. This process allows companies to implement their talent strategies more effectively.

[0342] (Example 2)

[0343] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0344] In modern recruitment practices, applicant evaluations are often based solely on superficial data, making it difficult to accurately assess an individual's true abilities and potential. In particular, the lack of evaluation that considers emotional insights and potential results in companies being unable to efficiently select the best talent.

[0345] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0346] In this invention, the server includes means for collecting personal information in various forms, means for extracting information from images using optical character recognition technology, means for analyzing the extracted text using natural language processing technology, means for analyzing an individual's emotional state using sentiment analysis technology, means for performing cross-analysis of the collected information using a generative AI model, means for evaluating an individual's creativity, technical skills, ethical values, and communication abilities, and means for predicting an individual's future performance and career growth potential based on these evaluations. This enables a comprehensive evaluation that takes into account not only superficial information but also an individual's potential and emotional insights.

[0347] "Various forms of personal information" refers to various data formats and types related to an individual, including information such as resumes, social media posts, image files, and document files.

[0348] "Optical character recognition technology" is a technology for extracting text data contained in images, handwritten documents, etc., as digital characters.

[0349] "Natural language processing technology" is a technology that analyzes the language that humans speak and write naturally and understands its meaning.

[0350] "Emotion analysis technology" is a technique that identifies emotions from text data and specifies emotional states such as positive, negative, or neutral.

[0351] A "generative AI model" is a group of algorithms that use artificial intelligence to generate and analyze data, and is used to analyze multiple data sources and evaluation factors in a unified manner.

[0352] Cross-analysis is a technique that analyzes interrelationships and trends by combining different datasets or evaluation factors.

[0353] An "evaluation profile" is a report that quantifies and visualizes an individual's various abilities and characteristics, and includes the results of their evaluation.

[0354] To implement this invention, a server, a terminal, and a user must work together. First, the user uses a terminal to provide the system with various forms of personal information, such as resumes, social media posts, and past works. The terminal converts this data into an appropriate format and sends it to the server.

[0355] The server uses the following technologies to process this personal information: it extracts text from image files using OCR technology, and analyzes the grammar and keywords of the extracted text using natural language processing (NLP) technology. It also analyzes the emotional state derived from the text using sentiment analysis technology to identify the individual's emotional state.

[0356] Next, the server uses a generative AI model to cross-analyze the collected personal information. This analysis evaluates an individual's creativity, technical skills, ethical values, and communication abilities, and generates an evaluation profile. Based on this profile, it predicts the individual's future performance and career growth potential.

[0357] For example, if an applicant is applying for a design position, they would upload a resume that includes data on how their past design work was created. The server would then use OCR technology to convert the resume image into text and NLP technology to analyze design-related skills and context. Furthermore, it would perform sentiment analysis on the applicant's social media posts to analyze their emotional state.

[0358] An example of a prompt using a generative AI model is "Please provide insights into the applicant's creative activities," which is then entered to start the analysis process. In this way, companies can obtain a comprehensive evaluation of applicants and efficiently select the most suitable talent.

[0359] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0360] Step 1:

[0361] Users input personal information such as resumes, social media posts, and past works into the system via their terminal. The input data consists of different formats (e.g., PDF, JPEG, text files). The terminal converts this data into a specific format and prepares it for transmission to the server.

[0362] Step 2:

[0363] The server receives personal information transmitted from the terminal. It then uses optical character recognition (OCR) to extract text data from image files. For example, it extracts text information from a scanned resume image to prepare for the next processing step.

[0364] Step 3:

[0365] The server utilizes natural language processing (NLP) technology to analyze the extracted text data. The input is text data from OCR, and it performs grammatical structure analysis and keyword extraction, determining the meaning based on that analysis. This forms the foundation for a deep understanding of user resumes and social media posts.

[0366] Step 4:

[0367] The server applies sentiment analysis techniques to identify the user's emotional state from text analyzed by NLP. For example, positive, negative, or neutral emotional states are identified within the context of a social media post. The resulting sentiment data is then used in subsequent cross-analysis.

[0368] Step 5:

[0369] The server uses a generative AI model to perform a complex cross-analysis using NLP analysis results and sentiment data as input. This analysis evaluates the user's creativity, technical skills, ethics, and communication abilities. An example of a prompt for the AI ​​model is, "Please provide insights into the applicant's creative activities."

[0370] Step 6:

[0371] The server generates an evaluation profile of the applicant based on the results of the cross-analysis. At this stage, each evaluation item is scored, and the user's characteristics are quantified. The generated profile is used in the next step.

[0372] Step 7:

[0373] Through the terminal, recruiters access a user interface that visualizes the generated evaluation profile. Graphs and charts displayed on the dashboard allow for a visual assessment of the user's abilities and potential. This provides companies with information to support their hiring decisions.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] In modern manufacturing environments, numerous machines operate in parallel, making their optimal placement and performance evaluation crucial. However, there is a lack of systems for objectively and comprehensively evaluating machine operation and performance. In particular, the inability to obtain intuitive understanding through emotional evaluations leads to decreased operational efficiency.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for collecting evaluation data in various formats, means for cross-analyzing the collected data using a generating AI, and means for emotionally evaluating the operating state using an emotion analysis engine. This makes it possible to improve the suitability evaluation and operational efficiency of machines in manufacturing sites.

[0379] "Evaluation target" is a general term for the objects analyzed by the system, and in this case, it refers to machines and robots operating within a factory.

[0380] "Generative AI" is a type of artificial intelligence technology that uses algorithms to analyze large amounts of data and derive relationships between them.

[0381] "Data collection means" refers to the technology or device used to collect necessary information from the subject of evaluation.

[0382] Cross-analysis is an analytical method that uses different perspectives and criteria to perform a multifaceted evaluation of collected data.

[0383] An "emotion analysis engine" is an analytical device or software that derives emotional meaning from the content and behavior of data.

[0384] The "integrated display surface" is an interface for displaying analysis results in an intuitively understandable format, and is provided as graphs and dashboards.

[0385] This system is designed to evaluate the suitability of machinery in a manufacturing environment and to enable efficient operation. The system is primarily composed of three components: a server, terminals, and users.

[0386] The server collects evaluation data in various formats and cross-analyzes it using generative AI. Specifically, the server extracts necessary text data from images using OCR technology and consolidates various operational data acquired from sensors into a unified format. Based on the collected data, the generative AI performs analysis to comprehensively evaluate the creativity, technical capabilities, and efficiency of machines and robots.

[0387] The emotion analysis engine analyzes video data and movement patterns, evaluating, for example, smooth movements as "positive emotions" and unnecessary movements or obstacles as "negative emotions." This analysis is performed using natural language processing such as IBM Watson Natural Language Understanding. The evaluation results are visualized on a dashboard, an integrated display surface, and provided to the user via the terminal.

[0388] The terminals used include smartphones and head-mounted displays, allowing for visual confirmation of data and viewing of analysis results. This information enables users to make appropriate decisions based on machine data in the manufacturing environment.

[0389] For example, consider a factory robot A that efficiently streamlines line work with its rapid movements and optimizes power consumption. This evaluation is reflected as "high efficiency" on the dashboard. Conversely, if machine B frequently stops working, it will be evaluated as "low efficiency," and maintenance will be recommended.

[0390] An example of a prompt might be text such as, "Analyze the robot's motion and emotionally evaluate its efficiency and performance." This allows the manufacturing environment to achieve higher productivity and efficiency.

[0391] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0392] Step 1:

[0393] The server collects operational data from each machine in the factory. Through sensors and IoT devices, it acquires data such as operating time, failure frequency, and power consumption for each machine. This data is stored in a database and formatted into a unified format. The input is raw data from each machine, and the output is a formatted dataset.

[0394] Step 2:

[0395] The server uses OCR technology on the formatted dataset to extract necessary text information from the image data. Here, Tesseract OCR is used to obtain metrics from images of the machine's dashboard and main screen. The input is an image file, and the output is text data.

[0396] Step 3:

[0397] The server utilizes generative AI to perform cross-analysis based on the collected data. This analysis scores the creativity, technical capabilities, and efficiency of machines and robots. The input is formatted and OCR-processed data, and the output is a score list representing the analysis results.

[0398] Step 4:

[0399] The server uses an emotion analysis engine to emotionally evaluate the operating state of each machine. Utilizing IBM Watson Natural Language Understanding, it evaluates smoothness and efficiency of operation as "positive emotions" and analyzes malfunctions and shutdowns as "negative emotions." The input is machine operation data, and the output is an emotion evaluation score.

[0400] Step 5:

[0401] The terminal visualizes the analysis results received from the server and displays them on a dashboard. Users can then use this to intuitively understand the machine's operating status and efficiency. The input is scored evaluation data, and the output is a visualized dashboard.

[0402] Step 6:

[0403] Users can review machine operation and layout on the factory floor based on visualized information. They can also determine the need for maintenance based on the analysis results. The input is evaluation information displayed on the dashboard, and the output is the user's decisions and actions.

[0404] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0405] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0406] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0407] [Third Embodiment]

[0408] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0409] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0410] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0411] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0412] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0413] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0414] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0415] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0416] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0417] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0418] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0419] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0420] This invention is a system for comprehensively evaluating applicants and predicting their future performance in a company's recruitment process. This system is implemented through a series of processes involving a server, terminals, and users.

[0421] First, users use their devices to provide the system with various forms of data about themselves. This includes uploading data through online forms and sharing links to social media accounts. The server collects this data and retrieves external data, including social media posts, using appropriate APIs. This centralizes the collection of information related to applicants.

[0422] Next, the server prepares to process data of different formats in a unified manner for analysis. OCR technology is used for text extraction from image data, and natural language processing technology is used for analyzing text data. After these preparations are complete, the server performs cross-analysis of the data using generative AI to extract the characteristics and skills of applicants.

[0423] After analysis, the server evaluates the applicant's creativity, technical skills, ethics, and communication abilities. AI scoring is applied to each evaluation item, and an applicant profile is generated based on these scores. This profile is displayed on a dashboard accessible to recruiters via their devices. The dashboard visualizes the evaluation results and strengths and weaknesses, allowing users to quickly understand the applicant's characteristics.

[0424] Furthermore, the server uses a model to predict future performance and plots and visualizes the career growth of applicants. This allows companies to make selections that take long-term talent strategies into consideration. For example, if a user applies for a design position, their design sense and speed of skill acquisition are evaluated and predicted based on their past work and submissions, and their future potential is presented. Through this series of processes, the present invention helps companies clearly understand the overall picture of applicants and select the most suitable talent.

[0425] The following describes the processing flow.

[0426] Step 1:

[0427] Users enter and submit application information using their devices. This includes uploading various files, such as resume data, links to social media accounts, and digital copies of past papers and works.

[0428] Step 2:

[0429] The server uses links provided by the user to retrieve relevant data from social media and other external resources via APIs. This completes the collection of all data related to the applicant.

[0430] Step 3:

[0431] The server formats the data. Specifically, it uses OCR to extract text from images and converts different data formats (e.g., text, images, documents) into a format that can be parsed.

[0432] Step 4:

[0433] The server runs a generative AI model to analyze the formatted data. Here, natural language processing is used to analyze textual data, and image analysis techniques are used to evaluate visual data.

[0434] Step 5:

[0435] Based on the analysis results, the server scores applicants' creativity, technical skills, ethical values, and communication abilities. This data is then used to generate an evaluation profile of the applicant.

[0436] Step 6:

[0437] On their devices, recruiters can visually review applicant evaluation results via a dashboard. The dashboard graphs the scored results, clearly displaying strengths and weaknesses.

[0438] Step 7:

[0439] The server uses a predictive model to forecast applicants' future performance and career growth. This information is also displayed on the dashboard.

[0440] Step 8:

[0441] Users and recruiters will use the information gathered to select the most suitable candidates and make decisions based on their long-term talent strategy.

[0442] (Example 1)

[0443] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0444] In traditional talent selection processes, it is difficult to effectively utilize diverse forms of applicant data and comprehensively evaluate candidates based on criteria such as creativity, technical skills, and ethical values. Furthermore, there is a lack of means to predict future performance and understand career growth. This makes it difficult for companies to select appropriate talent.

[0445] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0446] In this invention, the server includes means for collecting and centrally managing data in various formats from applicants, means for pre-processing information to extract information from images and text, and means for analyzing the collected information using a generative model to extract the characteristics of the applicants. This enables a multifaceted analysis of applicant information, allowing for evaluation based on creativity, technical ability, ethical values, and communication skills, as well as prediction of future job suitability and career growth.

[0447] An "applicant" is an individual who expresses interest in a job or role offered by a company or organization and submits information about it.

[0448] "Data in diverse formats" refers to all information provided in different formats and media, such as text, images, social media posts, and resume information.

[0449] "Centralized management" refers to the process of integrating data from different formats and sources to make it easier to handle within a single system.

[0450] "Preprocessing" refers to initial information processing activities to convert data into an analyzable format, and specifically includes text extraction from images and format conversion of text data.

[0451] A "generative model" refers to an algorithm or structure that uses artificial intelligence technology to learn features from large amounts of data and perform analysis and prediction of new information.

[0452] "Applicant characteristics" refer to distinctive data and information regarding the skills, abilities, values, and other characteristics of each individual applicant.

[0453] "Creativity" refers to the ability to create new value based on existing information and resources.

[0454] "Technical evaluation" refers to the activity of assessing the specialized knowledge and technical skills that applicants possess.

[0455] "Ethical values" refer to the moral judgment and values ​​that an applicant demonstrates, meaning they possess the right standards for judgment and action.

[0456] "Communication skills" refer to the ability to effectively exchange information with others and clearly convey one's intentions.

[0457] "Job suitability" refers to characteristics that indicate how adaptable and talented an individual is to a particular job or task.

[0458] "Job growth" refers to the predicted future scenario in which an applicant is likely to develop and grow in their career.

[0459] This system is designed to comprehensively evaluate applicants in a company's recruitment process and predict their future performance. The system is implemented through a process involving servers, terminals, and users.

[0460] Users use their devices to enter the data required for their application. This data includes information entered into online forms and the submission of links to their social media accounts. The server collects data from users and uses appropriate APIs to import external data, including social media posts, to centrally aggregate information related to applicants.

[0461] The server preprocesses the collected data. For image data, it uses OCR technology to extract text, and applies natural language processing techniques to the text data. Tools and frameworks such as TensorFlow and OpenCV are used to perform this processing.

[0462] The server uses a generative AI model to perform cross-analysis on the aggregated data. This analysis can identify characteristics such as applicants' creativity, technical skills, ethical values, and communication abilities. Based on the evaluated data, predictions are made about their future job suitability and career growth.

[0463] The analysis results and predictions are visualized on a dashboard generated on the server, which recruiters can access via their devices. The dashboard displays the evaluated results in graphs and charts, providing the information necessary for hiring decisions.

[0464] For example, when a user applies for a design position, the server analyzes their past design work and relevant social media posts to evaluate and predict their design sense and learning speed. Through this process, companies can gain a clear overall understanding of the applicant and receive support in selecting the right talent.

[0465] An example of a prompt is: "Evaluate the applicant's creativity and learning speed in a design role, and predict their future performance."

[0466] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0467] Step 1:

[0468] Users use their devices to enter the necessary data for their application into an online form. This data includes contact information, work experience, educational background, and social media account information. This information is entered into the system for the first time and is sent to the server.

[0469] Step 2:

[0470] The server initiates a process to collect and centrally manage data received from users. Utilizing SNS APIs, it retrieves relevant posting data from provided accounts and adds it to the system as already acquired information. This process integrates information from multiple data sources to form a complete applicant profile.

[0471] Step 3:

[0472] The server uses OCR and natural language processing technologies to preprocess the collected data. The input for this step is image and text data; OCR extracts text from images. Natural language processing tokenizes the text data and identifies important words such as nouns and verbs, preparing it for subsequent analysis.

[0473] Step 4:

[0474] After preprocessing, the server performs analysis using a generative AI model. This step reveals the characteristics of the applicants, such as creativity and technical skills. The input to this analysis is the output data from step 3, and the output is a profile of each applicant's characteristics. The analysis process applies machine learning algorithms to find correlations between the data.

[0475] Step 5:

[0476] The server predicts the applicant's future job suitability and career growth based on the analysis results. Using the trait profile obtained in Step 4 as input, a machine learning model simulates the future career path. The output of this step is a report showing the applicant's predicted growth curve and suitability.

[0477] Step 6:

[0478] The server generates a dashboard to visually display the obtained analysis results and predictions. In this step, the output data from step 5 is used as input and visualized in graph and chart format. The output is a visualized dashboard that the user can view through their terminal.

[0479] Step 7:

[0480] Users access the dashboard from their devices to view all information about applicants. Based on this information, users can gain data-driven insights to support decision-making regarding recruitment and talent strategy.

[0481] (Application Example 1)

[0482] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] In corporate recruitment activities, there is a need to quickly and accurately evaluate applicants' abilities and characteristics, and to support long-term decisions that enhance their future job performance and suitability for the organization. Furthermore, security risk assessment is also crucial, and there is a need for a system that can comprehensively perform these tasks.

[0484] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0485] In this invention, the server includes a device for aggregating applicant information in various formats, a device for analyzing the aggregated information using a generating AI, and a device for determining security clearance based on the applicant data. This enables a multifaceted evaluation of applicant characteristics and comprehensive support for talent selection, including security risks.

[0486] "Applicant information in various formats" refers to information in various data formats provided by applicants, including text data, image data, and online profile information.

[0487] "Aggregation device" refers to hardware or software for centrally collecting and integrating applicant information in different formats.

[0488] "Generative AI" refers to artificial intelligence technology used to extract and analyze the characteristics of applicants based on given data.

[0489] "Analysis device" refers to hardware or software that performs the necessary computational processing to analyze the collected applicant information.

[0490] "Creativity, technical skills, ethics, and communication skills" are indicators used to evaluate the characteristics of applicants, and include the novelty of deliverables, technical skills, moral judgment, and interpersonal communication skills.

[0491] A "device for predicting future abilities and career growth" refers to hardware or software that predicts an applicant's future work performance and career advancement based on their past data and characteristics.

[0492] "Security clearance" refers to an indicator that determines the level of information an applicant is permitted to access within an organization, based on background checks and character assessments.

[0493] A "display platform" refers to an interface that visually displays analysis results, allowing users to easily understand the characteristics of applicants.

[0494] "A device that supports the selection of organizational members" refers to hardware or software that selects the most suitable personnel based on analysis and prediction results, and supports recruitment activities.

[0495] The system that implements this application consists of a server, terminals, and users. The server plays the main role in information processing, aggregating applicant information in various formats and analyzing that data. Specifically, the server collects information from text data, image data, and online profiles obtained from applicants, extracts text from images using Tesseract OCR, and performs natural language processing on the text data using the spaCy library. Through this series of data processing, the characteristics and skills of applicants are extracted. After further analysis of the extracted data using generative AI technology, the applicant's future abilities and career growth are predicted, and the results are stored in the cloud and used as indicators to support corporate decision-making.

[0496] The terminal functions as an interface for providing applicant evaluation results to HR personnel. The evaluation results are visualized via a display platform, allowing personnel to refer to them and understand the applicant's characteristics. Users can access this service via smartphones or computers, enter applicant data, and view the resulting generated profiles and security clearance information.

[0497] As a concrete example, when a company hires a new security officer, there is a process to quickly determine the security clearance level based on the applicant's social media data and skill information. The prompt input to the generating AI model is in the format of, "Please perform an appropriate security assessment based on the data provided by this applicant," which prompts the AI ​​to start the analysis.

[0498] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0499] Step 1:

[0500] The server receives text and image data of applicants submitted by users. It takes data uploaded via online forms and social media links as input and organizes it into a format for centralized storage in a database on the server as output. This data is then converted to an appropriate format for subsequent analysis.

[0501] Step 2:

[0502] The server extracts text from image data using Tesseract OCR. The input is the image data collected in step 1, and the output is the extracted text data. In this data conversion process, the character information contained in the image is converted into text and sent to the next step.

[0503] Step 3:

[0504] The server uses the spaCy library to analyze text data. The input is the text data obtained in the previous step, and the output is information about the applicant's characteristics and traits. Through natural language processing, meaningful information is extracted from the text, and the applicant's characteristics are identified.

[0505] Step 4:

[0506] The server cross-analyzes the data analyzed using a generative AI model. Here, the characteristics of each applicant are input into the model. The generative AI model starts processing with the prompt message "Please perform an appropriate security assessment based on the data provided by this applicant," and outputs a comprehensive report including the applicant's security clearance.

[0507] Step 5:

[0508] The terminal displays the analysis results sent from the server. The input is the analysis report from the server, and the output is a dashboard that the user can visually review. The terminal graphically represents the evaluation results of characteristics and clearance levels, allowing the person in charge to easily evaluate applicants.

[0509] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0510] This invention is a system for a company's recruitment activities that comprehensively evaluates applicant data and predicts future performance. Furthermore, by incorporating an emotion engine that recognizes user emotions and utilizes that data for analysis, the accuracy of the evaluation is improved.

[0511] First, users provide the system with various forms of data about themselves using their devices. This data includes resumes, social media posts, and past works. The server collects this data and, if necessary, uses APIs to retrieve additional information from external data sources.

[0512] Next, the server preprocesses the data. It unifies the data format by extracting text from images using OCR technology and analyzing the sentences using natural language processing. During this process, the sentiment engine analyzes the user's emotions from the collected texts and social media posts, identifying positive, negative, and neutral emotions.

[0513] Next, a cross-analysis using generative AI is performed to evaluate the applicant's creativity, technical skills, ethics, and communication abilities. Emotional data obtained by the emotion engine is used to enhance the evaluation of communication abilities in particular. For example, the tone of the applicant's statements on social media is analyzed to provide additional insights into their interpersonal skills.

[0514] The server generates an evaluation profile of the applicant based on each analyzed evaluation element. This evaluation profile is displayed on a dashboard that recruiters access via their terminals. The dashboard visualizes the scored items, clearly showing the applicant's characteristics.

[0515] Furthermore, the server uses a predictive model to calculate the applicant's future performance and career growth potential. This predictive information is also reflected in the dashboard, which companies can use as a reference to build long-term talent strategies. For example, for an applicant aspiring to a design position, their design sense is evaluated based on their past work, and their past challenges and feelings of praise are analyzed through an emotion engine to predict their future potential.

[0516] In this way, this system evaluates applicants from multiple perspectives, helping companies efficiently select the most suitable personnel.

[0517] The following describes the processing flow.

[0518] Step 1:

[0519] Users submit their application data to the system using their devices. This process includes uploading resumes and past work files, and providing links to their social media accounts.

[0520] Step 2:

[0521] The server collects data from users and, if necessary, uses APIs to retrieve publicly available information from social networking platforms. This allows for the acquisition of additional data from users' online activities.

[0522] Step 3:

[0523] The server formats the collected data into a processable format. Specifically, it applies OCR technology to extract text from images and performs natural language processing on document data to analyze its content.

[0524] Step 4:

[0525] The emotion engine is activated to analyze the emotions contained in the user's text data and social media posts. This identifies and records positive, negative, or neutral emotions from the user's posts.

[0526] Step 5:

[0527] The server uses a generative AI model to perform analysis and evaluate applicants' creativity, technical skills, ethics, and communication abilities. Emotional data obtained from an emotion engine is used to add depth to the evaluation, particularly of communication skills.

[0528] Step 6:

[0529] The server generates applicant profiles based on the evaluation results. The evaluated scores are incorporated into a dashboard and displayed visually. Users and company recruiters can access this information to review the evaluation details.

[0530] Step 7:

[0531] The server uses a predictive model to forecast applicants' future performance and career growth. This forecast includes collected data and sentiment data, which are visualized on a dashboard.

[0532] Step 8:

[0533] Recruiters use a dashboard on their devices to make decisions about selecting the best candidates, based on an overall evaluation of applicants and their future projections. This process allows companies to implement their talent strategies more effectively.

[0534] (Example 2)

[0535] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0536] In modern recruitment practices, applicant evaluations are often based solely on superficial data, making it difficult to accurately assess an individual's true abilities and potential. In particular, the lack of evaluation that considers emotional insights and potential results in companies being unable to efficiently select the best talent.

[0537] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0538] In this invention, the server includes means for collecting personal information in various forms, means for extracting information from images using optical character recognition technology, means for analyzing the extracted text using natural language processing technology, means for analyzing an individual's emotional state using sentiment analysis technology, means for performing cross-analysis of the collected information using a generative AI model, means for evaluating an individual's creativity, technical skills, ethical values, and communication abilities, and means for predicting an individual's future performance and career growth potential based on these evaluations. This enables a comprehensive evaluation that takes into account not only superficial information but also an individual's potential and emotional insights.

[0539] "Various forms of personal information" refers to various data formats and types related to an individual, including information such as resumes, social media posts, image files, and document files.

[0540] "Optical character recognition technology" is a technology for extracting text data contained in images, handwritten documents, etc., as digital characters.

[0541] "Natural language processing technology" is a technology that analyzes the language that humans speak and write naturally and understands its meaning.

[0542] "Emotion analysis technology" is a technique that identifies emotions from text data and specifies emotional states such as positive, negative, or neutral.

[0543] A "generative AI model" is a group of algorithms that use artificial intelligence to generate and analyze data, and is used to analyze multiple data sources and evaluation factors in a unified manner.

[0544] Cross-analysis is a technique that analyzes interrelationships and trends by combining different datasets or evaluation factors.

[0545] An "evaluation profile" is a report that quantifies and visualizes an individual's various abilities and characteristics, and includes the results of their evaluation.

[0546] To implement this invention, a server, a terminal, and a user must work together. First, the user uses a terminal to provide the system with various forms of personal information, such as resumes, social media posts, and past works. The terminal converts this data into an appropriate format and sends it to the server.

[0547] The server uses the following technologies to process this personal information: it extracts text from image files using OCR technology, and analyzes the grammar and keywords of the extracted text using natural language processing (NLP) technology. It also analyzes the emotional state derived from the text using sentiment analysis technology to identify the individual's emotional state.

[0548] Next, the server uses a generative AI model to cross-analyze the collected personal information. This analysis evaluates an individual's creativity, technical skills, ethical values, and communication abilities, and generates an evaluation profile. Based on this profile, it predicts the individual's future performance and career growth potential.

[0549] For example, if an applicant is applying for a design position, they would upload a resume that includes data on how their past design work was created. The server would then use OCR technology to convert the resume image into text and NLP technology to analyze design-related skills and context. Furthermore, it would perform sentiment analysis on the applicant's social media posts to analyze their emotional state.

[0550] An example of a prompt using a generative AI model is "Please provide insights into the applicant's creative activities," which is then entered to start the analysis process. In this way, companies can obtain a comprehensive evaluation of applicants and efficiently select the most suitable talent.

[0551] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0552] Step 1:

[0553] Users input personal information such as resumes, social media posts, and past works into the system via their terminal. The input data consists of different formats (e.g., PDF, JPEG, text files). The terminal converts this data into a specific format and prepares it for transmission to the server.

[0554] Step 2:

[0555] The server receives personal information transmitted from the terminal. It then uses optical character recognition (OCR) to extract text data from image files. For example, it extracts text information from a scanned resume image to prepare for the next processing step.

[0556] Step 3:

[0557] The server utilizes natural language processing (NLP) technology to analyze the extracted text data. The input is text data from OCR, and it performs grammatical structure analysis and keyword extraction, determining the meaning based on that analysis. This forms the foundation for a deep understanding of user resumes and social media posts.

[0558] Step 4:

[0559] The server applies sentiment analysis techniques to identify the user's emotional state from text analyzed by NLP. For example, positive, negative, or neutral emotional states are identified within the context of a social media post. The resulting sentiment data is then used in subsequent cross-analysis.

[0560] Step 5:

[0561] The server uses a generative AI model to perform a complex cross-analysis using NLP analysis results and sentiment data as input. This analysis evaluates the user's creativity, technical skills, ethics, and communication abilities. An example of a prompt for the AI ​​model is, "Please provide insights into the applicant's creative activities."

[0562] Step 6:

[0563] The server generates an evaluation profile of the applicant based on the results of the cross-analysis. At this stage, each evaluation item is scored, and the user's characteristics are quantified. The generated profile is used in the next step.

[0564] Step 7:

[0565] Through the terminal, recruiters access a user interface that visualizes the generated evaluation profile. Graphs and charts displayed on the dashboard allow for a visual assessment of the user's abilities and potential. This provides companies with information to support their hiring decisions.

[0566] (Application Example 2)

[0567] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0568] In modern manufacturing environments, numerous machines operate in parallel, making their optimal placement and performance evaluation crucial. However, there is a lack of systems for objectively and comprehensively evaluating machine operation and performance. In particular, the inability to obtain intuitive understanding through emotional evaluations leads to decreased operational efficiency.

[0569] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0570] In this invention, the server includes means for collecting evaluation data in various formats, means for cross-analyzing the collected data using a generating AI, and means for emotionally evaluating the operating state using an emotion analysis engine. This makes it possible to improve the suitability evaluation and operational efficiency of machines in manufacturing sites.

[0571] "Evaluation target" is a general term for the objects analyzed by the system, and in this case, it refers to machines and robots operating within a factory.

[0572] "Generative AI" is a type of artificial intelligence technology that uses algorithms to analyze large amounts of data and derive relationships between them.

[0573] "Data collection means" refers to the technology or device used to collect necessary information from the subject of evaluation.

[0574] Cross-analysis is an analytical method that uses different perspectives and criteria to perform a multifaceted evaluation of collected data.

[0575] An "emotion analysis engine" is an analytical device or software that derives emotional meaning from the content and behavior of data.

[0576] The "integrated display surface" is an interface for displaying analysis results in an intuitively understandable format, and is provided as graphs and dashboards.

[0577] This system is designed to evaluate the suitability of machinery in a manufacturing environment and to enable efficient operation. The system is primarily composed of three components: a server, terminals, and users.

[0578] The server collects evaluation data in various formats and cross-analyzes it using generative AI. Specifically, the server extracts necessary text data from images using OCR technology and consolidates various operational data acquired from sensors into a unified format. Based on the collected data, the generative AI performs analysis to comprehensively evaluate the creativity, technical capabilities, and efficiency of machines and robots.

[0579] The emotion analysis engine analyzes video data and movement patterns, evaluating, for example, smooth movements as "positive emotions" and unnecessary movements or obstacles as "negative emotions." This analysis is performed using natural language processing such as IBM Watson Natural Language Understanding. The evaluation results are visualized on a dashboard, an integrated display surface, and provided to the user via the terminal.

[0580] The terminals used include smartphones and head-mounted displays, allowing for visual confirmation of data and viewing of analysis results. This information enables users to make appropriate decisions based on machine data in the manufacturing environment.

[0581] For example, consider a factory robot A that efficiently streamlines line work with its rapid movements and optimizes power consumption. This evaluation is reflected as "high efficiency" on the dashboard. Conversely, if machine B frequently stops working, it will be evaluated as "low efficiency," and maintenance will be recommended.

[0582] An example of a prompt might be text such as, "Analyze the robot's motion and emotionally evaluate its efficiency and performance." This allows the manufacturing environment to achieve higher productivity and efficiency.

[0583] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0584] Step 1:

[0585] The server collects operational data from each machine in the factory. Through sensors and IoT devices, it acquires data such as operating time, failure frequency, and power consumption for each machine. This data is stored in a database and formatted into a unified format. The input is raw data from each machine, and the output is a formatted dataset.

[0586] Step 2:

[0587] The server uses OCR technology on the formatted dataset to extract necessary text information from the image data. Here, Tesseract OCR is used to obtain metrics from images of the machine's dashboard and main screen. The input is an image file, and the output is text data.

[0588] Step 3:

[0589] The server utilizes generative AI to perform cross-analysis based on the collected data. This analysis scores the creativity, technical capabilities, and efficiency of machines and robots. The input is formatted and OCR-processed data, and the output is a score list representing the analysis results.

[0590] Step 4:

[0591] The server uses an emotion analysis engine to emotionally evaluate the operating state of each machine. Utilizing IBM Watson Natural Language Understanding, it evaluates smoothness and efficiency of operation as "positive emotions" and analyzes malfunctions and shutdowns as "negative emotions." The input is machine operation data, and the output is an emotion evaluation score.

[0592] Step 5:

[0593] The terminal visualizes the analysis results received from the server and displays them on a dashboard. Users can then use this to intuitively understand the machine's operating status and efficiency. The input is scored evaluation data, and the output is a visualized dashboard.

[0594] Step 6:

[0595] Users can review machine operation and layout on the factory floor based on visualized information. They can also determine the need for maintenance based on the analysis results. The input is evaluation information displayed on the dashboard, and the output is the user's decisions and actions.

[0596] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0597] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0599] [Fourth Embodiment]

[0600] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0601] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0602] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0604] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0606] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0607] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0608] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0609] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0612] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0613] This invention is a system for comprehensively evaluating applicants and predicting their future performance in a company's recruitment process. This system is implemented through a series of processes involving a server, terminals, and users.

[0614] First, users use their devices to provide the system with various forms of data about themselves. This includes uploading data through online forms and sharing links to social media accounts. The server collects this data and retrieves external data, including social media posts, using appropriate APIs. This centralizes the collection of information related to applicants.

[0615] Next, the server prepares to process data of different formats in a unified manner for analysis. OCR technology is used for text extraction from image data, and natural language processing technology is used for analyzing text data. After these preparations are complete, the server performs cross-analysis of the data using generative AI to extract the characteristics and skills of applicants.

[0616] After analysis, the server evaluates the applicant's creativity, technical skills, ethics, and communication abilities. AI scoring is applied to each evaluation item, and an applicant profile is generated based on these scores. This profile is displayed on a dashboard accessible to recruiters via their devices. The dashboard visualizes the evaluation results and strengths and weaknesses, allowing users to quickly understand the applicant's characteristics.

[0617] Furthermore, the server uses a model to predict future performance and plots and visualizes the career growth of applicants. This allows companies to make selections that take long-term talent strategies into consideration. For example, if a user applies for a design position, their design sense and speed of skill acquisition are evaluated and predicted based on their past work and submissions, and their future potential is presented. Through this series of processes, the present invention helps companies clearly understand the overall picture of applicants and select the most suitable talent.

[0618] The following describes the processing flow.

[0619] Step 1:

[0620] Users enter and submit application information using their devices. This includes uploading various files, such as resume data, links to social media accounts, and digital copies of past papers and works.

[0621] Step 2:

[0622] The server uses links provided by the user to retrieve relevant data from social media and other external resources via APIs. This completes the collection of all data related to the applicant.

[0623] Step 3:

[0624] The server formats the data. Specifically, it uses OCR to extract text from images and converts different data formats (e.g., text, images, documents) into a format that can be parsed.

[0625] Step 4:

[0626] The server runs a generative AI model to analyze the formatted data. Here, natural language processing is used to analyze textual data, and image analysis techniques are used to evaluate visual data.

[0627] Step 5:

[0628] Based on the analysis results, the server scores applicants' creativity, technical skills, ethical values, and communication abilities. This data is then used to generate an evaluation profile of the applicant.

[0629] Step 6:

[0630] On their devices, recruiters can visually review applicant evaluation results via a dashboard. The dashboard graphs the scored results, clearly displaying strengths and weaknesses.

[0631] Step 7:

[0632] The server uses a predictive model to forecast applicants' future performance and career growth. This information is also displayed on the dashboard.

[0633] Step 8:

[0634] Users and recruiters will use the information gathered to select the most suitable candidates and make decisions based on their long-term talent strategy.

[0635] (Example 1)

[0636] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0637] In traditional talent selection processes, it is difficult to effectively utilize diverse forms of applicant data and comprehensively evaluate candidates based on criteria such as creativity, technical skills, and ethical values. Furthermore, there is a lack of means to predict future performance and understand career growth. This makes it difficult for companies to select appropriate talent.

[0638] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0639] In this invention, the server includes means for collecting and centrally managing data in various formats from applicants, means for pre-processing information to extract information from images and text, and means for analyzing the collected information using a generative model to extract the characteristics of the applicants. This enables a multifaceted analysis of applicant information, allowing for evaluation based on creativity, technical ability, ethical values, and communication skills, as well as prediction of future job suitability and career growth.

[0640] An "applicant" is an individual who expresses interest in a job or role offered by a company or organization and submits information about it.

[0641] "Data in diverse formats" refers to all information provided in different formats and media, such as text, images, social media posts, and resume information.

[0642] "Centralized management" refers to the process of integrating data from different formats and sources to make it easier to handle within a single system.

[0643] "Preprocessing" refers to initial information processing activities to convert data into an analyzable format, and specifically includes text extraction from images and format conversion of text data.

[0644] A "generative model" refers to an algorithm or structure that uses artificial intelligence technology to learn features from large amounts of data and perform analysis and prediction of new information.

[0645] "Applicant characteristics" refer to distinctive data and information regarding the skills, abilities, values, and other characteristics of each individual applicant.

[0646] "Creativity" refers to the ability to create new value based on existing information and resources.

[0647] "Technical evaluation" refers to the activity of assessing the specialized knowledge and technical skills that applicants possess.

[0648] "Ethical values" refer to the moral judgment and values ​​that an applicant demonstrates, meaning they possess the right standards for judgment and action.

[0649] "Communication skills" refer to the ability to effectively exchange information with others and clearly convey one's intentions.

[0650] "Job suitability" refers to characteristics that indicate how adaptable and talented an individual is to a particular job or task.

[0651] "Job growth" refers to the predicted future scenario in which an applicant is likely to develop and grow in their career.

[0652] This system is designed to comprehensively evaluate applicants in a company's recruitment process and predict their future performance. The system is implemented through a process involving servers, terminals, and users.

[0653] Users use their devices to enter the data required for their application. This data includes information entered into online forms and the submission of links to their social media accounts. The server collects data from users and uses appropriate APIs to import external data, including social media posts, to centrally aggregate information related to applicants.

[0654] The server preprocesses the collected data. For image data, it uses OCR technology to extract text, and applies natural language processing techniques to the text data. Tools and frameworks such as TensorFlow and OpenCV are used to perform this processing.

[0655] The server uses a generative AI model to perform cross-analysis on the aggregated data. This analysis can identify characteristics such as applicants' creativity, technical skills, ethical values, and communication abilities. Based on the evaluated data, predictions are made about their future job suitability and career growth.

[0656] The analysis results and predictions are visualized on a dashboard generated on the server, which recruiters can access via their devices. The dashboard displays the evaluated results in graphs and charts, providing the information necessary for hiring decisions.

[0657] For example, when a user applies for a design position, the server analyzes their past design work and relevant social media posts to evaluate and predict their design sense and learning speed. Through this process, companies can gain a clear overall understanding of the applicant and receive support in selecting the right talent.

[0658] An example of a prompt is: "Evaluate the applicant's creativity and learning speed in a design role, and predict their future performance."

[0659] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0660] Step 1:

[0661] Users use their devices to enter the necessary data for their application into an online form. This data includes contact information, work experience, educational background, and social media account information. This information is entered into the system for the first time and is sent to the server.

[0662] Step 2:

[0663] The server initiates a process to collect and centrally manage data received from users. Utilizing SNS APIs, it retrieves relevant posting data from provided accounts and adds it to the system as already acquired information. This process integrates information from multiple data sources to form a complete applicant profile.

[0664] Step 3:

[0665] The server uses OCR and natural language processing technologies to preprocess the collected data. The input for this step is image and text data; OCR extracts text from images. Natural language processing tokenizes the text data and identifies important words such as nouns and verbs, preparing it for subsequent analysis.

[0666] Step 4:

[0667] After preprocessing, the server performs analysis using a generative AI model. This step reveals the characteristics of the applicants, such as creativity and technical skills. The input to this analysis is the output data from step 3, and the output is a profile of each applicant's characteristics. The analysis process applies machine learning algorithms to find correlations between the data.

[0668] Step 5:

[0669] The server predicts the applicant's future job suitability and career growth based on the analysis results. Using the trait profile obtained in Step 4 as input, a machine learning model simulates the future career path. The output of this step is a report showing the applicant's predicted growth curve and suitability.

[0670] Step 6:

[0671] The server generates a dashboard to visually display the obtained analysis results and predictions. In this step, the output data from step 5 is used as input and visualized in graph and chart format. The output is a visualized dashboard that the user can view through their terminal.

[0672] Step 7:

[0673] Users access the dashboard from their devices to view all information about applicants. Based on this information, users can gain data-driven insights to support decision-making regarding recruitment and talent strategy.

[0674] (Application Example 1)

[0675] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] In corporate recruitment activities, there is a need to quickly and accurately evaluate applicants' abilities and characteristics, and to support long-term decisions that enhance their future job performance and suitability for the organization. Furthermore, security risk assessment is also crucial, and there is a need for a system that can comprehensively perform these tasks.

[0677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0678] In this invention, the server includes a device for aggregating applicant information in various formats, a device for analyzing the aggregated information using a generating AI, and a device for determining security clearance based on the applicant data. This enables a multifaceted evaluation of applicant characteristics and comprehensive support for talent selection, including security risks.

[0679] "Applicant information in various formats" refers to information in various data formats provided by applicants, including text data, image data, and online profile information.

[0680] "Aggregation device" refers to hardware or software for centrally collecting and integrating applicant information in different formats.

[0681] "Generative AI" refers to artificial intelligence technology used to extract and analyze the characteristics of applicants based on given data.

[0682] "Analysis device" refers to hardware or software that performs the necessary computational processing to analyze the collected applicant information.

[0683] "Creativity, technical skills, ethics, and communication skills" are indicators used to evaluate the characteristics of applicants, and include the novelty of deliverables, technical skills, moral judgment, and interpersonal communication skills.

[0684] A "device for predicting future abilities and career growth" refers to hardware or software that predicts an applicant's future work performance and career advancement based on their past data and characteristics.

[0685] "Security clearance" refers to an indicator that determines the level of information an applicant is permitted to access within an organization, based on background checks and character assessments.

[0686] A "display platform" refers to an interface that visually displays analysis results, allowing users to easily understand the characteristics of applicants.

[0687] "A device that supports the selection of organizational members" refers to hardware or software that selects the most suitable personnel based on analysis and prediction results, and supports recruitment activities.

[0688] The system that implements this application consists of a server, terminals, and users. The server plays the main role in information processing, aggregating applicant information in various formats and analyzing that data. Specifically, the server collects information from text data, image data, and online profiles obtained from applicants, extracts text from images using Tesseract OCR, and performs natural language processing on the text data using the spaCy library. Through this series of data processing, the characteristics and skills of applicants are extracted. After further analysis of the extracted data using generative AI technology, the applicant's future abilities and career growth are predicted, and the results are stored in the cloud and used as indicators to support corporate decision-making.

[0689] The terminal functions as an interface for providing applicant evaluation results to HR personnel. The evaluation results are visualized via a display platform, allowing personnel to refer to them and understand the applicant's characteristics. Users can access this service via smartphones or computers, enter applicant data, and view the resulting generated profiles and security clearance information.

[0690] As a concrete example, when a company hires a new security officer, there is a process to quickly determine the security clearance level based on the applicant's social media data and skill information. The prompt input to the generating AI model is in the format of, "Please perform an appropriate security assessment based on the data provided by this applicant," which prompts the AI ​​to start the analysis.

[0691] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0692] Step 1:

[0693] The server receives text and image data of applicants submitted by users. It takes data uploaded via online forms and social media links as input and organizes it into a format for centralized storage in a database on the server as output. This data is then converted to an appropriate format for subsequent analysis.

[0694] Step 2:

[0695] The server extracts text from image data using Tesseract OCR. The input is the image data collected in step 1, and the output is the extracted text data. In this data conversion process, the character information contained in the image is converted into text and sent to the next step.

[0696] Step 3:

[0697] The server uses the spaCy library to analyze text data. The input is the text data obtained in the previous step, and the output is information about the applicant's characteristics and traits. Through natural language processing, meaningful information is extracted from the text, and the applicant's characteristics are identified.

[0698] Step 4:

[0699] The server cross-analyzes the data analyzed using a generative AI model. Here, the characteristics of each applicant are input into the model. The generative AI model starts processing with the prompt message "Please perform an appropriate security assessment based on the data provided by this applicant," and outputs a comprehensive report including the applicant's security clearance.

[0700] Step 5:

[0701] The terminal displays the analysis results sent from the server. The input is the analysis report from the server, and the output is a dashboard that the user can visually review. The terminal graphically represents the evaluation results of characteristics and clearance levels, allowing the person in charge to easily evaluate applicants.

[0702] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0703] This invention is a system for a company's recruitment activities that comprehensively evaluates applicant data and predicts future performance. Furthermore, by incorporating an emotion engine that recognizes user emotions and utilizes that data for analysis, the accuracy of the evaluation is improved.

[0704] First, users provide the system with various forms of data about themselves using their devices. This data includes resumes, social media posts, and past works. The server collects this data and, if necessary, uses APIs to retrieve additional information from external data sources.

[0705] Next, the server preprocesses the data. It unifies the data format by extracting text from images using OCR technology and analyzing the sentences using natural language processing. During this process, the sentiment engine analyzes the user's emotions from the collected texts and social media posts, identifying positive, negative, and neutral emotions.

[0706] Next, a cross-analysis using generative AI is performed to evaluate the applicant's creativity, technical skills, ethics, and communication abilities. Emotional data obtained by the emotion engine is used to enhance the evaluation of communication abilities in particular. For example, the tone of the applicant's statements on social media is analyzed to provide additional insights into their interpersonal skills.

[0707] The server generates an evaluation profile of the applicant based on each analyzed evaluation element. This evaluation profile is displayed on a dashboard that recruiters access via their terminals. The dashboard visualizes the scored items, clearly showing the applicant's characteristics.

[0708] Furthermore, the server uses a predictive model to calculate the applicant's future performance and career growth potential. This predictive information is also reflected in the dashboard, which companies can use as a reference to build long-term talent strategies. For example, for an applicant aspiring to a design position, their design sense is evaluated based on their past work, and their past challenges and feelings of praise are analyzed through an emotion engine to predict their future potential.

[0709] In this way, this system evaluates applicants from multiple perspectives, helping companies efficiently select the most suitable personnel.

[0710] The following describes the processing flow.

[0711] Step 1:

[0712] Users submit their application data to the system using their devices. This process includes uploading resumes and past work files, and providing links to their social media accounts.

[0713] Step 2:

[0714] The server collects data from users and, if necessary, uses APIs to retrieve publicly available information from social networking platforms. This allows for the acquisition of additional data from users' online activities.

[0715] Step 3:

[0716] The server formats the collected data into a processable format. Specifically, it applies OCR technology to extract text from images and performs natural language processing on document data to analyze its content.

[0717] Step 4:

[0718] The emotion engine is activated to analyze the emotions contained in the user's text data and social media posts. This identifies and records positive, negative, or neutral emotions from the user's posts.

[0719] Step 5:

[0720] The server uses a generative AI model to perform analysis and evaluate applicants' creativity, technical skills, ethics, and communication abilities. Emotional data obtained from an emotion engine is used to add depth to the evaluation, particularly of communication skills.

[0721] Step 6:

[0722] The server generates applicant profiles based on the evaluation results. The evaluated scores are incorporated into a dashboard and displayed visually. Users and company recruiters can access this information to review the evaluation details.

[0723] Step 7:

[0724] The server uses a predictive model to forecast applicants' future performance and career growth. This forecast includes collected data and sentiment data, which are visualized on a dashboard.

[0725] Step 8:

[0726] Recruiters use a dashboard on their devices to make decisions about selecting the best candidates, based on an overall evaluation of applicants and their future projections. This process allows companies to implement their talent strategies more effectively.

[0727] (Example 2)

[0728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] In modern recruitment practices, applicant evaluations are often based solely on superficial data, making it difficult to accurately assess an individual's true abilities and potential. In particular, the lack of evaluation that considers emotional insights and potential results in companies being unable to efficiently select the best talent.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0731] In this invention, the server includes means for collecting personal information in various forms, means for extracting information from images using optical character recognition technology, means for analyzing the extracted text using natural language processing technology, means for analyzing an individual's emotional state using sentiment analysis technology, means for performing cross-analysis of the collected information using a generative AI model, means for evaluating an individual's creativity, technical skills, ethical values, and communication abilities, and means for predicting an individual's future performance and career growth potential based on these evaluations. This enables a comprehensive evaluation that takes into account not only superficial information but also an individual's potential and emotional insights.

[0732] "Various forms of personal information" refers to various data formats and types related to an individual, including information such as resumes, social media posts, image files, and document files.

[0733] "Optical character recognition technology" is a technology for extracting text data contained in images, handwritten documents, etc., as digital characters.

[0734] "Natural language processing technology" is a technology that analyzes the language that humans speak and write naturally and understands its meaning.

[0735] "Emotion analysis technology" is a technique that identifies emotions from text data and specifies emotional states such as positive, negative, or neutral.

[0736] A "generative AI model" is a group of algorithms that use artificial intelligence to generate and analyze data, and is used to analyze multiple data sources and evaluation factors in a unified manner.

[0737] Cross-analysis is a technique that analyzes interrelationships and trends by combining different datasets or evaluation factors.

[0738] An "evaluation profile" is a report that quantifies and visualizes an individual's various abilities and characteristics, and includes the results of their evaluation.

[0739] To implement this invention, a server, a terminal, and a user must work together. First, the user uses a terminal to provide the system with various forms of personal information, such as resumes, social media posts, and past works. The terminal converts this data into an appropriate format and sends it to the server.

[0740] The server uses the following technologies to process this personal information: it extracts text from image files using OCR technology, and analyzes the grammar and keywords of the extracted text using natural language processing (NLP) technology. It also analyzes the emotional state derived from the text using sentiment analysis technology to identify the individual's emotional state.

[0741] Next, the server uses a generative AI model to cross-analyze the collected personal information. This analysis evaluates an individual's creativity, technical skills, ethical values, and communication abilities, and generates an evaluation profile. Based on this profile, it predicts the individual's future performance and career growth potential.

[0742] For example, if an applicant is applying for a design position, they would upload a resume that includes data on how their past design work was created. The server would then use OCR technology to convert the resume image into text and NLP technology to analyze design-related skills and context. Furthermore, it would perform sentiment analysis on the applicant's social media posts to analyze their emotional state.

[0743] An example of a prompt using a generative AI model is "Please provide insights into the applicant's creative activities," which is then entered to start the analysis process. In this way, companies can obtain a comprehensive evaluation of applicants and efficiently select the most suitable talent.

[0744] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0745] Step 1:

[0746] Users input personal information such as resumes, social media posts, and past works into the system via their terminal. The input data consists of different formats (e.g., PDF, JPEG, text files). The terminal converts this data into a specific format and prepares it for transmission to the server.

[0747] Step 2:

[0748] The server receives personal information transmitted from the terminal. It then uses optical character recognition (OCR) to extract text data from image files. For example, it extracts text information from a scanned resume image to prepare for the next processing step.

[0749] Step 3:

[0750] The server utilizes natural language processing (NLP) technology to analyze the extracted text data. The input is text data from OCR, and it performs grammatical structure analysis and keyword extraction, determining the meaning based on that analysis. This forms the foundation for a deep understanding of user resumes and social media posts.

[0751] Step 4:

[0752] The server applies sentiment analysis techniques to identify the user's emotional state from text analyzed by NLP. For example, positive, negative, or neutral emotional states are identified within the context of a social media post. The resulting sentiment data is then used in subsequent cross-analysis.

[0753] Step 5:

[0754] The server uses a generative AI model to perform a complex cross-analysis using NLP analysis results and sentiment data as input. This analysis evaluates the user's creativity, technical skills, ethics, and communication abilities. An example of a prompt for the AI ​​model is, "Please provide insights into the applicant's creative activities."

[0755] Step 6:

[0756] The server generates an evaluation profile of the applicant based on the results of the cross-analysis. At this stage, each evaluation item is scored, and the user's characteristics are quantified. The generated profile is used in the next step.

[0757] Step 7:

[0758] Through the terminal, recruiters access a user interface that visualizes the generated evaluation profile. Graphs and charts displayed on the dashboard allow for a visual assessment of the user's abilities and potential. This provides companies with information to support their hiring decisions.

[0759] (Application Example 2)

[0760] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] In modern manufacturing environments, numerous machines operate in parallel, making their optimal placement and performance evaluation crucial. However, there is a lack of systems for objectively and comprehensively evaluating machine operation and performance. In particular, the inability to obtain intuitive understanding through emotional evaluations leads to decreased operational efficiency.

[0762] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0763] In this invention, the server includes means for collecting evaluation data in various formats, means for cross-analyzing the collected data using a generating AI, and means for emotionally evaluating the operating state using an emotion analysis engine. This makes it possible to improve the suitability evaluation and operational efficiency of machines in manufacturing sites.

[0764] "Evaluation target" is a general term for the objects analyzed by the system, and in this case, it refers to machines and robots operating within a factory.

[0765] "Generative AI" is a type of artificial intelligence technology that uses algorithms to analyze large amounts of data and derive relationships between them.

[0766] "Data collection means" refers to the technology or device used to collect necessary information from the subject of evaluation.

[0767] Cross-analysis is an analytical method that uses different perspectives and criteria to perform a multifaceted evaluation of collected data.

[0768] An "emotion analysis engine" is an analytical device or software that derives emotional meaning from the content and behavior of data.

[0769] The "integrated display surface" is an interface for displaying analysis results in an intuitively understandable format, and is provided as graphs and dashboards.

[0770] This system is designed to evaluate the suitability of machinery in a manufacturing environment and to enable efficient operation. The system is primarily composed of three components: a server, terminals, and users.

[0771] The server collects evaluation data in various formats and cross-analyzes it using generative AI. Specifically, the server extracts necessary text data from images using OCR technology and consolidates various operational data acquired from sensors into a unified format. Based on the collected data, the generative AI performs analysis to comprehensively evaluate the creativity, technical capabilities, and efficiency of machines and robots.

[0772] The emotion analysis engine analyzes video data and movement patterns, evaluating, for example, smooth movements as "positive emotions" and unnecessary movements or obstacles as "negative emotions." This analysis is performed using natural language processing such as IBM Watson Natural Language Understanding. The evaluation results are visualized on a dashboard, an integrated display surface, and provided to the user via the terminal.

[0773] The terminals used include smartphones and head-mounted displays, allowing for visual confirmation of data and viewing of analysis results. This information enables users to make appropriate decisions based on machine data in the manufacturing environment.

[0774] For example, consider a factory robot A that efficiently streamlines line work with its rapid movements and optimizes power consumption. This evaluation is reflected as "high efficiency" on the dashboard. Conversely, if machine B frequently stops working, it will be evaluated as "low efficiency," and maintenance will be recommended.

[0775] An example of a prompt might be text such as, "Analyze the robot's motion and emotionally evaluate its efficiency and performance." This allows the manufacturing environment to achieve higher productivity and efficiency.

[0776] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0777] Step 1:

[0778] The server collects operational data from each machine in the factory. Through sensors and IoT devices, it acquires data such as operating time, failure frequency, and power consumption for each machine. This data is stored in a database and formatted into a unified format. The input is raw data from each machine, and the output is a formatted dataset.

[0779] Step 2:

[0780] The server uses OCR technology on the formatted dataset to extract necessary text information from the image data. Here, Tesseract OCR is used to obtain metrics from images of the machine's dashboard and main screen. The input is an image file, and the output is text data.

[0781] Step 3:

[0782] The server utilizes generative AI to perform cross-analysis based on the collected data. This analysis scores the creativity, technical capabilities, and efficiency of machines and robots. The input is formatted and OCR-processed data, and the output is a score list representing the analysis results.

[0783] Step 4:

[0784] The server uses an emotion analysis engine to emotionally evaluate the operating state of each machine. Utilizing IBM Watson Natural Language Understanding, it evaluates smoothness and efficiency of operation as "positive emotions" and analyzes malfunctions and shutdowns as "negative emotions." The input is machine operation data, and the output is an emotion evaluation score.

[0785] Step 5:

[0786] The terminal visualizes the analysis results received from the server and displays them on a dashboard. Users can then use this to intuitively understand the machine's operating status and efficiency. The input is scored evaluation data, and the output is a visualized dashboard.

[0787] Step 6:

[0788] Users can review machine operation and layout on the factory floor based on visualized information. They can also determine the need for maintenance based on the analysis results. The input is evaluation information displayed on the dashboard, and the output is the user's decisions and actions.

[0789] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0790] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0791] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0792] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0793] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0794] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0795] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0796] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0797] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0798] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0799] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0800] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0801] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0802] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0803] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0804] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0805] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0806] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0807] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0808] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0809] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0810] The following is further disclosed regarding the embodiments described above.

[0811] (Claim 1)

[0812] Means for collecting applicant data in various formats,

[0813] A method for cross-analyzing data collected using generative AI,

[0814] A means of evaluating applicants' creativity, technical skills, ethical values, and communication abilities,

[0815] Based on these evaluations, a means of predicting the future performance and career growth of applicants,

[0816] A system that includes this.

[0817] (Claim 2)

[0818] The system according to claim 1, comprising means for generating a dashboard for visualizing the analysis results of applicants.

[0819] (Claim 3)

[0820] The system according to claim 1, comprising means for supporting a company's personnel selection based on analysis and prediction results.

[0821] "Example 1"

[0822] (Claim 1)

[0823] A means of collecting and centrally managing data in various formats from applicants,

[0824] A means of preprocessing information to extract it from images and text,

[0825] A method for analyzing information collected using a generative model and extracting the characteristics of applicants,

[0826] Based on the extracted characteristics, a means of evaluating creativity, technical skills, ethics, and communication abilities,

[0827] Based on these evaluations, a means of predicting the applicant's future job suitability and career growth,

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, comprising a display device for visualizing analysis results and predictions.

[0831] (Claim 3)

[0832] The system according to claim 1, comprising means for supporting the organization's personnel selection based on analysis and prediction results.

[0833] "Application Example 1"

[0834] (Claim 1)

[0835] A device for aggregating applicant information in various formats,

[0836] A device that analyzes aggregated information using generative AI,

[0837] A device for evaluating applicants' creativity, technical skills, ethics, and communication abilities,

[0838] A device that estimates the applicant's future abilities and career growth based on these evaluations,

[0839] A device that determines security clearance based on applicant data,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, comprising a device for generating a display platform for visualizing the analysis results of applicants.

[0843] (Claim 3)

[0844] The system according to claim 1, comprising a device that supports the selection of organizational members based on analysis and prediction results.

[0845] "Example 2 of combining an emotion engine"

[0846] (Claim 1)

[0847] Means of collecting personal information in various forms,

[0848] A means of extracting information from an image using optical character recognition technology,

[0849] A means for analyzing text extracted using natural language processing technology,

[0850] A means of analyzing an individual's emotional state using emotion analysis technology,

[0851] A means of performing cross-analysis of information collected using a generative AI model,

[0852] A means of evaluating an individual's creativity, technical skills, ethical values, and communication abilities,

[0853] A means of predicting an individual's future performance and career growth potential based on these evaluations,

[0854] A system that includes this.

[0855] (Claim 2)

[0856] The system according to claim 1, comprising means for generating a user interface for visualizing analysis results and prediction results.

[0857] (Claim 3)

[0858] The system according to claim 1, comprising means for assisting an organization in selecting personnel based on analysis and prediction results.

[0859] "Application example 2 when combining with an emotional engine"

[0860] (Claim 1)

[0861] Means for collecting evaluation data in various formats,

[0862] A method for cross-analyzing data collected using generative AI,

[0863] A means of evaluating the creativity, skills, ethical standards, and communication abilities of the subject being assessed,

[0864] A means of predicting the future performance and growth potential of the subject under evaluation based on these assessments,

[0865] A means of emotionally evaluating the operating state using an emotion analysis engine,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, comprising means for generating an integrated display surface for visualizing the analysis results of the subject to evaluation.

[0869] (Claim 3)

[0870] The system according to claim 1, comprising means for providing support for selecting an institution based on analysis and prediction results. [Explanation of Symbols]

[0871] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting applicant data in various formats, A method for cross-analyzing data collected using generative AI, A means of evaluating applicants' creativity, technical skills, ethical values, and communication abilities, Based on these evaluations, a means of predicting the future performance and career growth of applicants, A system that includes this.

2. The system according to claim 1, comprising means for generating a dashboard for visualizing the analysis results of applicants.

3. The system according to claim 1, comprising means for supporting a company's personnel selection based on analysis and prediction results.

Images