Information processing system

Abstract

The invention provides an information processing system. The information processing system comprises a communication infrastructure arranged on a signal lamp; the data acquisition device is used for acquiring traffic video data; the data acquisition device is used for acquiring meteorological data; the data aggregation device is used for aggregating the data acquired by the data acquisition device through a high-speed communication network; the analysis device is used for analyzing the data sent by the data aggregation device; and the control device is used for controlling the signal lamp based on the analysis result of the analysis device.

Description

Technical Field

[0001] The technology disclosed herein relates to an information processing system. Background Technology

[0002] Japanese Patent Application Publication No. 2022-180282 discloses a method for controlling a role-based chatbot executed by at least one processor. The method includes the following steps: receiving a user's speech; adding the user's speech to a prompt word, the prompt word containing instruction statements associated with an explanation of the chatbot's role; encoding the prompt word; and inputting the encoded prompt word into a language model to generate a chatbot response to the user's speech.

[0003] With the expansion of urban traffic and the increase in the number of motor vehicles, existing traffic signal control systems are unable to respond in a timely and accurate manner to changes in traffic flow, sudden traffic accidents, traffic violations, and extreme weather (such as road icing and snow accumulation), leading to increased traffic congestion, frequent traffic accidents, and low management efficiency. There is an urgent need for a system that can dynamically collect and comprehensively analyze traffic and meteorological data to achieve intelligent signal control, thereby improving traffic safety and operational efficiency. Summary of the Invention

[0004] This invention proposes an information processing system, including communication infrastructure mounted on traffic lights, a data acquisition device for collecting traffic video data, a data acquisition device for collecting meteorological data, a data aggregation device for aggregating data via a high-speed communication network, an analysis device for parsing the aforementioned data, and a control device for dynamically controlling the traffic light status based on the analysis results. The analysis device can identify traffic flow, traffic accidents, vehicles violating traffic rules, and predict the risk of road icing and snow accumulation. The control device can dynamically adjust the traffic light display status according to the analysis results, achieving efficient traffic management for traffic congestion relief, accident emergency response, and severe weather conditions.

[0005] "Communication infrastructure" refers to the hardware and software components installed on traffic lights to enable high-speed data communication with external devices, including wireless communication modules, network interfaces, etc.

[0006] "Data acquisition device" refers to a device used to acquire traffic video data or meteorological data in real time, including equipment such as cameras and meteorological sensors.

[0007] "Traffic video data" refers to video information collected by cameras and other equipment that reflects the traffic conditions at intersections or on roads.

[0008] "Meteorological data" refers to data collected by meteorological sensors and other equipment, including temperature, humidity, wind speed, and other data that reflect the current road surface environment.

[0009] A "data aggregation device" refers to a device that centrally processes and summarizes data collected by multiple data acquisition devices through a high-speed communication network.

[0010] "High-speed communication network" refers to a wireless or wired network that can support high-capacity, low-latency data transmission, such as 5G network.

[0011] "Analysis device" refers to software or hardware used for intelligent analysis and processing of collected and aggregated data, including functions such as traffic flow recognition, license plate recognition, accident judgment, and meteorological risk assessment.

[0012] "Control device" refers to equipment that adjusts and controls the working status of traffic lights in real time based on data analysis results, including hardware such as traffic light controllers and corresponding control programs.

[0013] "Traffic lights" refer to electronically controlled display devices used at road intersections and other locations to indicate the passage and stop status of vehicles and pedestrians.

[0014] "Traffic violation vehicles" refers to motor vehicles that commit traffic violations such as speeding and running red lights while driving on the road.

[0015] "Road icing and snow accumulation" refers to the phenomenon where ice or snow forms on the road surface due to decreased temperature or snowfall, which may affect vehicle driving safety. Attached Figure Description

[0016] Figure 1 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the first embodiment.

[0017] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0018] Figure 3 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the second embodiment.

[0019] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0020] Figure 5 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the third embodiment.

[0021] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and head-mounted terminal according to the third embodiment.

[0022] Figure 7This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the fourth embodiment.

[0023] Figure 8 This is a conceptual diagram illustrating an example of the main functions of the data processing device and robot according to the fourth embodiment.

[0024] Figure 9 This represents an emotion map that maps multiple emotions.

[0025] Figure 10 This represents an emotion map that maps multiple emotions.

[0026] Figure 11 This is a sequence diagram illustrating the processing flow of the data processing system of the first embodiment.

[0027] Figure 12 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 1.

[0028] Figure 13 This is a sequence diagram illustrating the processing flow of the data processing system of the second embodiment.

[0029] Figure 14 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 2. Detailed Implementation

[0030] Hereinafter, an example of an implementation of the system according to the present disclosure will be described with reference to the accompanying drawings.

[0031] First, let me explain the terminology used in the following instructions.

[0032] In the following embodiments, the processor (hereinafter referred to as "processor") with reference numerals may be a single computing device or a combination of multiple computing devices. Furthermore, the processor may be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0033] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory that temporarily stores information and is used as working memory by the processor.

[0034] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disks (e.g., hard disks), or magnetic tapes.

[0035] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface that includes a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. As an example of a communication specification applicable to the communication I / F, wireless communication specifications such as 5G (5th Generation Mobile Communication System), Wi-Fi (wireless fidelity) (registered trademark), or Bluetooth (registered trademark) can be listed.

[0036] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects to express more than three items, the same interpretation as "A and / or B" applies.

[0037] First Implementation Method

[0038] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0039] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. A server can be cited as an example of the data processing device 12.

[0040] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0041] The smart device 14 includes a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiving device 38, output device 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0042] The receiving device 38 includes a touchscreen 38A and a microphone 38B, and receives user input. The touchscreen 38A receives user input via touch by detecting contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input via sound by detecting the user's voice. The control unit 46A in the processor 46 sends data representing the user input received by the touchscreen 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data representing the user input.

[0043] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting data in a form perceptible to the user 20 (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0044] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0045] Figure 2 The diagram shows an example of the main functions of the data processing device 12 and the smart device 14.

[0046] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0047] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0048] In the smart device 14, the processor 46 performs the acceptance output processing. The memory 50 stores the acceptance output program 60. The acceptance output program 60 is used in conjunction with the data processing system 10 and the specific processing program 56. The processor 46 reads the acceptance output program 60 from the memory 50 and executes the read acceptance output program 60 on the RAM 48. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48. Furthermore, the smart device 14 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48.

[0049] Alternatively, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-held terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing of the data processing system 10 of the first embodiment will be described.

[0050] Example 1

[0051] The flow of a specific process in Example 1 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. Furthermore, the data processing device 12 is referred to as the "server," and the smart device 14 is referred to as the "terminal."

[0052] Existing traffic management systems struggle to dynamically adjust traffic lights based on real-time traffic flow, traffic anomalies, and weather changes. They also fail to promptly identify high-risk events such as traffic accidents, traffic violations, and road icing or snow accumulation, leading to traffic congestion, frequent accidents, and low efficiency for emergency vehicles. Furthermore, current traffic visualization and remote control methods are insufficient, hindering relevant departments from responding promptly and efficiently.

[0053] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 1 is achieved by the following means.

[0054] In this invention, the server includes a data transceiver communication device, a vehicle and environmental information acquisition device, a data aggregation and processing device, and an intelligent analysis device and a signal control device built based on generative artificial intelligence models and optical character recognition technology. This enables efficient acquisition, processing, and AI-powered intelligent analysis of large amounts of traffic and meteorological data, achieving vehicle attribute identification, traffic flow estimation, traffic accident and violation detection, and prediction of road icing and snow accumulation risks. Based on the analysis results, it automatically and dynamically optimizes traffic light control status and provides users with visualized data and remote management support, thereby effectively alleviating traffic congestion and improving traffic safety and urban emergency response capabilities.

[0055] "Communication device" refers to a hardware or software module used on information display equipment to send and receive data.

[0056] "Imaging device" refers to a sensing device used to capture and acquire dynamic video or images of vehicles on the road.

[0057] "Sensor device" refers to a measuring device used to collect data on surrounding environmental conditions (including temperature, humidity, wind speed, etc.) in real time.

[0058] "Data acquisition device" refers to a collection of equipment, including imaging devices and sensor devices, used to collect information related to vehicles and the environment.

[0059] A "data consolidation device" refers to a device that acquires multi-source data through a high-speed communication network and performs unified integration, time-series correction, and packaging of the data.

[0060] "Data processing device" refers to the processing module that preprocesses and standardizes the information received from the data consolidation device.

[0061] "Information processing device" refers to a computing platform that performs high-performance data analysis and logical reasoning tasks, including artificial intelligence algorithms.

[0062] "Generative artificial intelligence model" refers to an artificial intelligence computing model that can perform feature analysis, event judgment, and data reasoning based on big data.

[0063] Optical character recognition technology refers to software or algorithms that automatically convert text in an image into computer-readable text information.

[0064] "Analysis device" refers to a data processing unit that uses generative artificial intelligence models and optical character recognition technology to analyze and detect collected data and output structured results.

[0065] "Signal display device" refers to a display and control terminal used to indicate and guide road traffic flow (such as traffic lights).

[0066] "Control device" refers to an automated decision-making unit that dynamically adjusts the working status of the signal display device based on the analysis results.

[0067] "Management support device" refers to a terminal or platform that visualizes, records, remotely displays, and assists in the management and control of traffic data and analysis results.

[0068] To facilitate the practice of this invention, the following detailed description of its embodiments, in conjunction with specific hardware and software systems, is provided.

[0069] The system described in this invention is mainly composed of three main components: a server, a terminal, and a user. The server is used for centralized data processing and intelligent judgment, the terminal is mainly responsible for front-end data acquisition and execution control, and the user remotely monitors and manages the entire system through an interactive interface.

[0070] The server can be deployed in data centers or cloud platforms and possesses high-performance computing capabilities. It is equipped with a data processing platform (such as Linux-based server hardware) and integrates generative artificial intelligence model processing engines (such as TensorFlow and PyTorch) as well as optical character recognition (OCR) software (such as Tesseract OCR). The server runs a relational database (such as MySQL) for result recording and management, and also supports web visualization platforms (such as Tableau or a custom WebDashboard) to display analysis and alert information to users.

[0071] The terminal mainly includes cameras, sensors, data aggregation devices, and signal control terminals. Common network cameras (such as general-purpose industrial cameras) can be used as imaging devices, while environmental sensor modules can be used for meteorological data acquisition. Each terminal maintains high-speed communication with the server via 5G or wired Ethernet. The terminal integrates an edge computing gateway (such as an edge computing industrial control computer) to perform local data integration and preliminary processing, and also includes traffic light control devices (such as traffic signal controller modules) to ensure that the analysis results can be physically applied to traffic control.

[0072] The user operating platform can be a laptop, desktop, or mobile terminal. Users can remotely access the web management system to view traffic conditions, analysis reports, and risk events in real time, and issue traffic management instructions as needed.

[0073] In practical applications, the terminal continuously collects raw data such as traffic videos and weather data, and uploads it to the server via the 5G network. The server first performs unified structuring and standardization of the data, and then calls a generative artificial intelligence model to perform vehicle recognition, traffic flow analysis, and accident monitoring on the image data. It also uses OCR to recognize license plate information and combines environmental data to conduct road risk assessment. All processing results are stored in a database, and key events are displayed to the user in a visual format.

[0074] Users can make timely response decisions based on real-time data and alarm records pushed by the server. For example, if a user discovers congestion at an intersection, they can remotely instruct the adjustment of traffic light timings or request traffic police to handle traffic violations on-site. The entire process involves a clear, automated, and efficient data flow and decision-making chain.

[0075] Specific examples

[0076] Taking a city intersection as an example, the terminal is equipped with a 30fps industrial camera and environmental sensors. The server uses TensorFlow to deploy AI models to complete traffic flow statistics and accident detection, and uses Tesseract OCR to extract license plate information. Based on environmental data of continuous cooling and rising humidity, the server identifies the risk of road icing and automatically issues traffic light control instructions: extending green lights to alleviate congestion, and shifting red lights to emergency lanes, etc. Users can see real-time data and reports through a visualization platform and promptly issue management decisions.

[0077] Example of specific prompts for generative artificial intelligence models:

[0078] 1. How exactly does the server use generative artificial intelligence models and OCR to automatically identify traffic accidents from uploaded videos and weather data? Please explain the processing flow in detail.

[0079] 2. When the terminal adjusts the traffic lights according to signal control instructions, what specific hardware operations and communication protocols are executed? Please describe in detail.

[0080] 3. What real-time data can users view and manage through the data visualization interface? Please provide examples from specific urban traffic management scenarios.

[0081] use Figure 11 The processing flow is explained.

[0082] Step 1:

[0083] The terminal collects traffic video data and meteorological data through cameras and environmental sensors. Inputs include real-time images of the road environment and meteorological parameters (such as temperature, humidity, and wind speed). The terminal timestamps the collected raw data and stores it in a local cache in real time. Outputs are time-synchronized video and meteorological data streams.

[0084] Step 2:

[0085] The terminal utilizes a data consolidation device to integrate, deduplicate, and align the acquired video and meteorological data over time. The input is the raw data synchronized from the camera and sensors. The terminal performs data format conversion and packaging on the multi-source data, outputting structured data packets for easy subsequent network transmission.

[0086] Step 3:

[0087] The terminal uses a 5G communication module to securely and efficiently upload the processed data packets to the data server. The input is integrated and packaged traffic and weather data; the terminal records and verifies data integrity during the upload process. The output is the raw data packets received by the server.

[0088] Step 4:

[0089] After receiving data packets from the terminal, the server first performs data integrity verification, format checks, and anomaly correction. The input is the data file uploaded by the terminal. The server standardizes the video and weather content into a standard format, excluding invalid segments or erroneous items from the data. The output is a formatted dataset without anomalies.

[0090] Step 5:

[0091] The server utilizes generative artificial intelligence models and optical character recognition (OCR) technology to perform deep intelligent analysis of the data. The input consists of formatted traffic video and weather data. The server uses AI models to analyze the video footage, identify vehicles, record traffic flow, and determine abnormal events (such as accidents and illegal parking). Simultaneously, it uses OCR to recognize license plate information and combines this with weather data to perform road risk assessments (such as icing and snow accumulation). The output is a structured event recognition result, including traffic volume, accident warnings, a list of vehicles violating traffic rules, and road risk levels.

[0092] Step 6:

[0093] The server generates signal control commands based on the AI ​​analysis results and sends them to the traffic light control device in the terminal. The input is the event recognition and analysis results. The server integrates traffic flow, accident conditions, and road risks to automatically calculate the optimal traffic light cycle and special traffic strategies. The output is the specific signal control commands.

[0094] Step 7:

[0095] The terminal's traffic light control device automatically adjusts the traffic light status based on received control commands. This includes extending green light time, switching to red light, or setting priority lanes for emergency vehicles. Input is the control commands sent from the server. Through physical relays and control modules, the terminal achieves real-time hardware operation of the traffic lights. Output is the status change of the traffic lights on site.

[0096] Step 8:

[0097] The server stores the analyzed data and control logs in a database and displays them to users in real time through a visualization platform. Inputs include AI analysis results and on-site feedback data. The server performs visualization processing on various data types, generating statistical reports and risk alerts. Output is an easy-to-understand visual interface.

[0098] Step 9:

[0099] Users can view real-time traffic flow, weather, and event data via the web or other management terminals, and issue traffic control commands based on system suggestions or their own judgment. Input includes a server-side visual dashboard and alarm information. Users directly click command buttons, and the system records these commands, achieving closed-loop management. Output is the actual traffic control or dispatch information following the user's decision.

[0100] Application Example 1

[0101] The process flow corresponding to the specific processing in Use Case 1 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. Furthermore, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0102] Existing traffic management systems suffer from lags in real-time acquisition and integration of traffic video and environmental meteorological data, failing to make rapid intelligent signal adjustments based on changes in road conditions and the environment. Furthermore, the lack of a comprehensive analysis and feedback mechanism for the biometric information and emotional responses of traffic participants hinders efficient early warning and dynamic command during emergencies such as traffic congestion and accidents. In addition, current systems cannot flexibly utilize generative artificial intelligence models to automatically generate personalized notifications for different audiences, reducing management efficiency and information reach. Therefore, there is an urgent need for a new generation of traffic control systems that integrate multi-source sensing, intelligent analysis, dynamic control, and intelligent notification generation to improve the safety and intelligence of traffic operations.

[0103] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is achieved by the following means.

[0104] In this invention, the server includes: an information aggregation device for collecting physical phenomenon data and environmental information data from sensing terminals at signal display devices; an information processing device for performing multi-dimensional information analysis and event recognition based on data collected from broadband networks; an information processing device for collecting and analyzing users' biometric and voice information, and inferring users' emotional states in real time through inference models; and an output processing device capable of automatically generating and outputting personalized notification content through a generative artificial intelligence model by combining the analysis results and user states. This enables: multi-dimensional real-time perception and efficient analysis of traffic conditions, environmental risks, and user emotions; automatic adjustment of signal displays based on comprehensive analysis results; optimization of object flow; improved emergency response efficiency for sudden events; and enhanced timeliness and accuracy of information services through a flexible and personalized information notification mechanism.

[0105] "Signal display device" refers to an electronic device used to display the status of traffic lights and manage and guide road traffic flow.

[0106] "Information and communication device" refers to hardware or its integrated components used to realize information data transmission and reception, signal transmission and network communication.

[0107] "Sensing device" refers to a sensor device that can collect data on physical phenomena and environmental information, including but not limited to cameras, weather sensors, etc.

[0108] "Physical phenomenon data" refers to real-time data that characterizes the state of objects on the road, traffic flow, and other related information.

[0109] "Environmental information data" refers to data on environmental conditions such as temperature, humidity, and wind speed related to traffic scenarios.

[0110] "Broadband information and communication network" refers to an information network system that enables high-capacity, high-speed data transmission, such as 4G / 5G communication networks.

[0111] "Information aggregation device" refers to a data processing device used to integrate and upload data from multiple sensing devices in a unified manner.

[0112] "Information processing device" refers to a computing device or its software system that performs functions such as data parsing, event recognition and analysis.

[0113] "Control device" refers to a device that regulates the state of a signal display device based on the analysis results of an information processing device.

[0114] "Bioinformation" refers to data generated by the human body to reflect physiological or psychological states, including but not limited to facial expressions and heart rate.

[0115] "Voice information" refers to audio data that reflects a user's emotions or intentions, obtained by collecting and analyzing a person's voice.

[0116] "User status" refers to the specific state of a user's psychological or emotional response inferred based on biometric and voice information.

[0117] "Inference model" refers to an algorithmic model used for intelligent analysis, inference, and prediction of input data, including artificial intelligence models.

[0118] "Generative AI models" refer to AI systems that can automatically generate text content or decision suggestions based on input information.

[0119] "Output processing device" refers to a device that can automatically send the generated notification content to the user or front-end device.

[0120] "Notification information" refers to prompts, warnings, or guidance text content generated by the system and output to a specific target audience.

[0121] "Rules violated in water bodies" refers to traffic participants who fail to drive or stay in accordance with regulations in traffic scenarios.

[0122] "Prompt statements" refer to instructions or texts that guide generative artificial intelligence models to generate specific outputs, as input content.

[0123] To facilitate understanding of the present invention, an embodiment of the present invention will be described in detail below with reference to specific structures and examples.

[0124] This invention relates to an intelligent traffic control system based on multi-source perception, intelligent analysis, and generative artificial intelligence. The system mainly includes an information communication device located at the signal display device, a sensing device for collecting physical phenomenon data and environmental information data, an information aggregation device for integrating data through a broadband information communication network, an information processing device for performing multi-dimensional information analysis and event inference, an information processing device for inferring user status based on user biometric information and voice information, an output processing device for automatically generating personalized notification information based on a generative artificial intelligence model, and a control device for dynamically controlling the signal display device.

[0125] The server can be deployed in the traffic management center to coordinate and manage various sensing terminals and perform large-scale data analysis and intelligent information generation. On the hardware side, sensing devices include, but are not limited to, high-definition cameras (such as network cameras), weather sensors (such as commonly used automatic weather stations), emotion-collecting microphones, and facial expression recognition cameras. The server can be a general-purpose server with a high-performance GPU and multi-core CPU. Information communication and aggregation devices support wired Ethernet and wireless broadband networks such as 5G to ensure real-time and high-bandwidth data interaction.

[0126] In terms of software implementation, the server can perform video stream traffic flow analysis using open-source computer vision tools (such as OpenCV), license plate recognition and traffic anomaly detection using text recognition modules (such as OCR software), and road icing or snow accumulation risk prediction using meteorological analysis models such as Pythonstatsmodels. Simultaneously, the server has a built-in emotion recognition engine, for example, using deep learning neural networks to classify facial expressions and speech data for emotion. The generative AI model used for generating notification content can be deployed using mainstream large language models or local APIs, automatically generating prompts, announcements, and management suggestions for traffic participants or managers based on actual needs.

[0127] During system operation, terminals (sensing devices) continuously collect traffic video, meteorological data, and user biometric and voice information in real time, and upload them to the server via the network. The server analyzes and processes the various data, dynamically makes decisions and generates control signals based on traffic conditions, environmental risks, user emotional changes, and management rules. These signals are then sent to traffic lights and other actuators via control devices to optimize traffic flow and respond to emergencies. Furthermore, the system can automatically push important events with personalized, structured content to smart terminals, mobile applications, or information panels. Users (such as traffic police or management agencies) can also remotely view, monitor, and adjust commands through a graphical interface.

[0128] Taking a practical application as an example, at a major intersection in a city, the terminal collects high-definition video, weather information, and facial expressions and voice data of pedestrians in real time. The server analyzes and identifies congestion, accidents, and emotional stress using an AI model, automatically extending the green light time, prioritizing light control, and generating structured notifications using a generative artificial intelligence model, such as: "Currently, it is peak hour with heavy traffic. Illegal overtaking has been detected, and some citizens are experiencing anxiety. Please remain patient. The green light time has been extended, and priority passage routes for emergency vehicles have been set." This notification is pushed in real time through multiple channels, including a mobile app and intersection broadcasts, effectively improving the intelligence of management and the citizen service experience.

[0129] Example of specific prompts for generative artificial intelligence models:

[0130] "Please automatically generate an intelligent traffic report based on the latest traffic, weather, and user sentiment data, along with traffic management suggestions."

[0131] Input: At 7:30 AM, at the intersection in the city center of A, there were 20 vehicles in traffic, the temperature was 2 degrees Celsius, the humidity was 80%, and 3 vehicles were speeding. Some users were feeling anxious.

[0132] Output: Please generate a detailed daily traffic monitoring report to provide optimization suggestions for traffic management personnel.

[0133] This invention provides an efficient and intelligent innovative technical solution for smart city traffic management by combining hardware and software in data acquisition, intelligent analysis, and notification output, enabling multi-source information fusion, autonomous traffic decision-making, and personalized notification services.

[0134] use Figure 12 The processing flow is explained.

[0135] Step 1:

[0136] The terminal collects real-time traffic video data from roads and intersections via cameras, environmental information such as temperature, humidity, and wind speed via weather sensors, and facial expressions and voice information from nearby users via an emotion capture device. The input consists of physical phenomenon data, environmental information data, and user biometric and voice information. The terminal performs preliminary formatting and data packet structuring on this raw data. The output is a structured multi-source data packet.

[0137] Step 2:

[0138] The terminal utilizes a broadband information communication network (such as 5G) to upload structured data packets to an information aggregation device on a server in real time. The input consists of various structured data packets. The data is securely encrypted during transmission and aggregated to a designated aggregation node. The output is a data stream that has been uploaded to the central server.

[0139] Step 3:

[0140] The server receives and stores data streams from multiple terminals. Input consists of multi-source data streams and data packets. The server performs format validation and preprocessing on video streams, environmental data, and sentiment data, removing invalid or abnormal data. Output is preprocessed, valid data stored in the database.

[0141] Step 4:

[0142] The server uses computer vision analysis tools to perform object detection and traffic flow analysis on traffic video data, and utilizes OCR technology for license plate recognition and traffic violation detection. The input is processed video stream data. Through image framing and feature extraction, the system identifies vehicle type, quantity, and license plate information, compares this information with a rule base, and outputs traffic flow statistics and data on vehicles committing violations.

[0143] Step 5:

[0144] The server uses meteorological data analysis models (such as the ARIMA model) to perform time-series analysis and trend forecasting on data such as temperature, humidity, and wind speed. The input is environmental information data. After inputting the data into the model, the server calculates the probability of road icing and snow accumulation. The output is the meteorological risk assessment result.

[0145] Step 6:

[0146] The server utilizes an emotion recognition engine (combining facial expression analysis and speech analysis models) to perform multimodal emotion recognition on user biometric and speech data. The input consists of user facial expression and speech data. After processing by a deep learning network, the output is an emotional state label for the user group (e.g., tense, anxious, calm).

[0147] Step 7:

[0148] The server integrates traffic flow, weather risk, user sentiment data, and traffic control rules to automatically generate signal control strategies suitable for the current situation. Inputs include various types of structured data obtained from the analysis and predefined rules. After algorithmic decision-making, the output is specific signal control instructions (such as extending green lights or setting priority passage).

[0149] Step 8:

[0150] The server sends signal control commands to the terminal (signal display device) in real time via the control device. The terminal switches the traffic light status according to the received commands and can push key traffic information in real time via APP or broadcast. The input is the signal controller control commands and notification content. The output is the dynamically changing signal display and the broadcast / push notification information.

[0151] Step 9:

[0152] The server utilizes a generative artificial intelligence model to automatically generate structured or natural language notification texts and management suggestions based on the above analysis results and the current traffic scenario. The inputs include analyzed data, prompts, and target user types. The server inputs the data as prompts into the generative AI model and outputs targeted, personalized notification, announcement, or management suggestion texts.

[0153] Step 10:

[0154] Users can remotely view real-time traffic data, weather information, user sentiment analysis results, and generated notifications through a data visualization interface or mobile device. When necessary, they can manually adjust traffic light strategies or issue specific management instructions based on system suggestions or actual observations. Input consists of visualized data and notifications pushed from the backend. Users make decisions or provide feedback based on the displayed content, and the output is the adjusted management plan or feedback information uploaded to the system.

[0155] Alternatively, an emotion engine for inferring user emotions can be combined. That is, the specific processing unit 290 can also use the emotion-specific model 59 to infer user emotions and perform specific processing using user emotions.

[0156] Example 2

[0157] The flow of a specific process in Example 2 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 as the "terminal."

[0158] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 2 is achieved by the following means.

[0159] In this invention, the server includes information communication infrastructure, an information acquisition device for obtaining dynamic image information of vehicles and pedestrians, an information acquisition device for obtaining environmental status information, an information aggregation device for integrating multi-source information, an information analysis device for analyzing the aggregated information, a control device for remote control based on the analysis results, an individual identification and violation status recognition device, an environmental change prediction device, and a psychological state analysis device, as well as an information processing device for user psychological state adjustment notifications and control strategies. This enables real-time acquisition, analysis, and fusion of multi-dimensional data such as traffic, environment, and user emotions, dynamically optimizing traffic light control and information push, effectively alleviating traffic pressure, improving public safety and user experience, and realizing comprehensive management and services for intelligent transportation.

[0160] "Information and communication infrastructure" refers to a data transmission platform that enables high-speed, high-capacity data communication between remote devices and a central system, including but not limited to wired networks, wireless networks, and related communication hardware facilities.

[0161] "Information acquisition device" refers to equipment used to collect data related to the external environment, including but not limited to camera equipment used to acquire dynamic image information of vehicles and pedestrians, and sensor equipment used to acquire environmental status information.

[0162] "Information collection device" refers to a device used to receive and integrate data from multiple information collection devices. After the data is packaged, synchronized and uniformly formatted, it facilitates subsequent information processing and analysis.

[0163] "Information analysis device" refers to a software system or computing platform that can intelligently analyze, extract features, and identify events from collected raw data.

[0164] "Control device" refers to a device that, based on the results of information analysis, makes real-time adjustments to the actions of remote control equipment such as signaling equipment and notification systems, and issues task instructions.

[0165] "Individual identification device" refers to a data analysis device used to identify the unique identity of traffic participants (such as vehicle identification, pedestrian facial features, etc.) and determine whether there is any violation.

[0166] "Environmental change prediction device" refers to software or hardware devices that perform trend analysis and risk prediction of changes in road surface (such as snow accumulation and icing) based on environmental status information.

[0167] A "psychological state analysis device" refers to an analysis system that automatically identifies a user's current psychological state (such as anxiety, stress, or calmness) by analyzing the user's image and voice information.

[0168] "Information processing device" refers to the hardware and software unit that automatically adjusts the notification push method or control strategy based on the results of psychological state analysis to ensure the realization of humanized and personalized services.

[0169] To facilitate the accurate implementation of this invention, the following describes in detail a specific implementation of an intelligent traffic management system in conjunction with the technical solution of the invention.

[0170] This system works collaboratively between terminals (such as smart terminal devices at traffic intersections) and servers (such as the backend server of a traffic control center) to achieve real-time collection, intelligent analysis, and dynamic control of urban traffic conditions, environmental conditions, and user psychological states. Terminals are responsible for data collection and basic data processing, while servers are responsible for in-depth fusion analysis and decision-making based on large-scale, multi-dimensional data. Users experience personalized services through receiving notifications or interactive interactions.

[0171] The core hardware of this system includes: camera equipment for capturing real-time dynamic images of vehicles and pedestrians at intersections and roads (such as industrial-grade network cameras); environmental sensors for acquiring environmental data such as temperature, humidity, and wind speed (such as multi-parameter meteorological sensors); audio and facial recognition equipment for acquiring audio and facial images of passing users (such as digital microphones and high-definition cameras); an edge computing unit for initial packaging and integration of multi-source raw data (such as embedded edge computing modules); and high-speed communication network components, such as 5G communication modules, to achieve high-speed and stable data transmission.

[0172] The system's software platform includes: information aggregation software, which formats, encrypts, and synchronizes multi-source collected data; an information analysis module, based on a generative artificial intelligence model, which comprehensively uses AI algorithms such as object detection, OCR text recognition, behavior analysis, and environmental change prediction (e.g., YOLOv5, PaddleOCR, TensorFlow, OpenCV, etc.); a psychological state analysis module, which uses deep learning networks to perform multimodal emotion recognition on user facial expressions and voice information (e.g., emotion recognition API, Librosa tool, Face++); dynamic control software, which automatically generates traffic light scheduling schemes and message push strategies for different traffic and user psychological scenarios; and an information processing unit, which personalizes and adjusts notification content and interaction methods based on server analysis results (e.g., intelligent traffic controller, user terminal APP).

[0173] During system operation, the terminal collects traffic, environmental and user-related data in real time through the aforementioned hardware, performs preliminary data processing using the edge computing unit, and uploads the data to the server via the 5G network. The server simultaneously aggregates and intelligently analyzes a large amount of intersection data to achieve the following functions: (1) Identify traffic flow, traffic accidents, traffic violations and other abnormal events; (2) Predict road risks such as snow accumulation and icing based on environmental data; (3) Analyze the user's psychological state based on real-time collected user facial expressions and voice, such as tension, anxiety, and anger, and combine the analysis results with traffic management decisions to dynamically adjust signal control and information notification content.

[0174] These decisions are fed back to the terminal and directly affect traffic lights, traffic guidance screens, or user terminals, enabling intelligent and personalized traffic management and information delivery.

[0175] Specific examples

[0176] For example, during the morning rush hour on a city's main roads, the terminal's camera equipment acquires traffic flow information at intersections at a rate of 30 frames per second, while environmental sensors detect an air temperature of 2°C and humidity of 80%. The system automatically analyzes the video stream, identifies the length of vehicle queues, and uses OCR technology to determine if any vehicles are violating traffic rules. When users upload facial expressions and voice recordings via a mobile app, the system analyzes their psychological state, concluding that some drivers are experiencing anxiety. The server integrates traffic, environmental, and user emotional information to dynamically extend green light durations, adjust notification formats, and push messages such as "The weather is slippery; your passage time at the intersection has been extended" to key users.

[0177] Example of prompts for generative artificial intelligence models:

[0178] "Please provide a detailed explanation of the working principle of the real-time traffic management system based on the 5G communication network, with particular emphasis on the methods for collecting traffic videos, meteorological data, and user sentiment data, and for AI-powered intelligent analysis."

[0179] "How can we achieve adaptive control of city traffic lights and personalized information delivery to users by integrating multimodal sensing and artificial intelligence?"

[0180] use Figure 13 The processing flow is explained.

[0181] Step 1:

[0182] The terminal utilizes a camera, weather sensors, and audio acquisition devices to collect real-time traffic dynamic images, environmental parameters (such as temperature, humidity, and wind speed), and user facial expressions and voice information. Inputs include raw video streams from the intersection, weather sensor data, and raw voice / facial image data. The terminal performs preliminary formatting and time stamping on the various data types, outputting structured data packets with timestamps. Specific actions include the camera capturing vehicle and pedestrian movements at 30 frames per second, the weather sensor measuring environmental data every minute, and the microphone and camera simultaneously acquiring user audio and facial expression images.

[0183] Step 2:

[0184] The terminal performs initial packaging and aggregation of the collected multi-source data through the edge computing unit, and then uploads it to the server via the 5G communication module. The input is a structured data packet, and data processing includes data integration, encryption, and data packet reformatting. The output is a data packet that can be transmitted over the network. Specifically, the edge computing unit aggregates all real-time data, and the 5G module efficiently uploads the packaged data to the traffic management server.

[0185] Step 3:

[0186] After receiving multi-dimensional data uploaded by the terminal, the server first calls the traffic AI module to perform traffic flow analysis, target detection, and violation recognition. The input includes video image data and meteorological parameter data. The server uses technologies such as YOLOv5 for vehicle and pedestrian detection, and OCR or AI models (such as PaddleOCR) for license plate and other feature recognition, comparing the results with a traffic rules database. Data processing includes target detection, feature extraction, and behavior recognition. The output includes analytical results such as traffic volume, violation types, and accident warnings. Specifically, the server automatically processes all uploaded image data periodically every few seconds to detect traffic density and abnormal behavior.

[0187] Step 4:

[0188] The server analyzes meteorological data to assess potential environmental risks such as snow accumulation and icing on roads. The input consists of multi-point meteorological parameters, which are processed by meteorological analysis algorithms (such as rule-based judgment or machine learning prediction models). The output includes road condition anomaly warnings and risk levels. Specifically, the server makes judgments based on minimum temperature, humidity, and wind speed thresholds, and combines this with historical data to predict short-term risks.

[0189] Step 5:

[0190] The server invokes the psychological state analysis module to perform emotion recognition on user voice and facial expression images uploaded from the terminal. The input consists of user facial data and audio data. Data processing includes facial feature point detection and voice feature analysis, using emotion recognition algorithms (such as TensorFlow, Face++ API, etc.) to determine the user's emotions (e.g., tension, anger, anxiety). The output is the labeled user psychological state. Specifically, the server automatically batch processes user data collected at different time periods, adding emotion tags to each user.

[0191] Step 6:

[0192] The server integrates all the above analysis results, calls traffic intelligent control algorithms, and generates traffic light schemes and information push strategies. Inputs include analysis results of traffic flow, traffic violations, weather risks, and user psychological states. Based on a rule engine and AI optimization algorithms, the server outputs dynamic signal control parameters and personalized notification content. Specifically, it automatically generates traffic light duration adjustment instructions, sends them to the on-site control system, and simultaneously integrates user status to generate corresponding message templates.

[0193] Step 7:

[0194] The terminal controls the local traffic light status and display information according to server interface instructions, and pushes necessary messages to the user's terminal APP and mini-program. Input consists of control instructions and notification content returned by the server. Data processing includes traffic light time settings, message pushes, and display content refreshes. Output is real-time traffic light control and personalized information pushes. Specifically, the terminal automatically switches signal colors, changes display content, and sends real-time safety alerts or personalized information to the user terminal based on new instructions.

[0195] Application Example 2

[0196] The process flow corresponding to the specific processing in Use Case 2 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. In addition, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0197] Existing traffic management systems typically rely on single types of data for analysis when dealing with traffic congestion, accidents, and adverse weather conditions. They lack the comprehensive perception and processing of traffic conditions, road environment, and the real-time emotional states of traffic participants (such as drivers and passengers). Consequently, they struggle to achieve intelligent dynamic control of traffic signal equipment and autonomous vehicles, and cannot provide personalized information feedback based on individual psychological and physiological states, resulting in limited improvements in overall traffic efficiency and safety.

[0198] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is achieved by the following means.

[0199] In this invention, the server includes a device for collecting traffic status and environmental information via a communication medium, an observation device for collecting individual identification and physiological information, an information aggregation device for data aggregation via a high-speed communication path, a computational processing device for performing multivariate analysis of the data, a control device for generating and issuing control signals based on the analysis results, and a notification output device for providing personalized information based on the analysis and physiological information. This allows for comprehensive analysis of traffic flow, road condition risks, traffic violations, and the psychological state of traffic participants. Based on these analysis results, signal equipment and autonomous driving equipment can be dynamically and efficiently controlled, while adaptive and personalized safety prompts and information feedback are provided to individuals, thereby effectively improving the intelligence level and traffic safety of the traffic management system.

[0200] "Communication medium" refers to a physical or wireless channel used for data transmission between signal equipment and observation devices, information aggregation devices, and servers, such as optical fiber, cable, or wireless network.

[0201] "Traffic status information" refers to data that reflects the traffic flow, vehicle types, pedestrian activities, traffic accidents, and other conditions on the road.

[0202] "Environmental information" refers to data related to the road and traffic environment, such as weather, temperature, humidity, and ground conditions.

[0203] "Observation device" refers to data collection equipment used to collect traffic status information, environmental information, individual identification information and physiological information in real time.

[0204] "Individual identification information" refers to relevant data that can identify traffic participants, such as facial images, voice features, or identification codes.

[0205] "Physiological information" refers to data that reflects the current physical or emotional state of traffic participants, including facial expressions, tone of voice, heart rate, etc.

[0206] "Information aggregation device" refers to a data processing unit that can aggregate data from multiple observation devices and transmit it to a server via a high-speed communication path.

[0207] "High-speed communication path" refers to a communication network that supports high-capacity, high-speed data transmission, such as 5G mobile network or wired broadband network.

[0208] "Computational processing unit" refers to an information processing unit that has the ability to analyze, reason, and process integrated data, such as a server or a dedicated computing platform.

[0209] "Control device" refers to the hardware that automatically generates and outputs control signals based on the analysis results to adjust signal equipment and autonomous driving equipment.

[0210] "Signal equipment" refers to signal indications used in traffic situations, including traffic lights, signs, and other traffic control devices.

[0211] "Autonomous driving equipment" refers to vehicles or means of transportation that can automatically adjust their driving strategies based on control signals.

[0212] "Notification output device" refers to an application or terminal device that outputs personalized information or prompts based on an individual's physiological and psychological state, such as in-vehicle terminals and mobile devices.

[0213] This invention can be implemented in the following ways.

[0214] This system employs terminal devices installed at intersections or transportation hubs. These terminals include traffic condition monitoring devices (such as high-definition cameras), environmental information monitoring devices (such as meteorological sensors, including temperature, humidity, and wind speed sensors), and cameras and microphones for collecting individual identification and physiological information. Each terminal is connected to the information aggregation device via high-speed communication paths such as 5G, enabling high-capacity, low-latency data transmission.

[0215] All raw data collected by the terminal is first formatted and preliminarily filtered locally using an edge computing module (such as an embedded hardware platform like NVIDIA Jetson) to ensure the efficiency and accuracy of the data transmitted to the server. The acquisition and preprocessing software used can be custom-developed based on general frameworks such as OpenCV and PyAudio. The preliminarily processed data is then uploaded to an information aggregation device via a high-speed communication network and finally uploaded to the server.

[0216] The server, acting as a computing and processing device, employs a high-performance computing platform (such as general-purpose server hardware or cloud computing resources) and integrates with an AI analysis engine to perform subsequent data analysis. The AI ​​analysis engine can be built on AI development platforms such as TensorFlow and PyTorch, and includes functional modules such as multimodal fusion, object detection, trajectory analysis, license plate recognition, meteorological data analysis, and real-time emotion recognition. Facial expression recognition and voice emotion analysis can be integrated with open-source libraries or commercial APIs (such as Baidu UNIT emotion analysis and OpenCV face recognition). Unified data management and processing can utilize database systems such as MySQL or MongoDB.

[0217] After analysis, the server automatically generates control commands for signal equipment and autonomous driving devices based on the integrated information. For example, in cases of peak traffic congestion, extreme weather, or high user stress, the server automatically adjusts traffic light timings, prioritizes specific directions of traffic, or issues commands to autonomous vehicles to slow down or change lanes based on AI model results. The server can also customize personalized notification content and provide feedback to users through various channels such as mobile terminals, in-vehicle screens, and voice broadcasts, indicating current road conditions and travel suggestions.

[0218] Taking a practical application as an example, during the city's morning rush hour, the terminal can capture intersection video at 30 frames per second, collect real-time weather and user facial expression data, and the server uses AI analysis to determine traffic congestion risks, predict road icing, and dynamically adjust traffic light modes and push personalized reminders based on user stress levels. This system can respond to external changes within seconds, effectively ensuring traffic safety and the mental well-being of personnel.

[0219] Here are some examples of prompts for generative artificial intelligence models:

[0220] Please simulate an urban traffic system. Inputs include real-time traffic camera video, weather sensor data, and user facial expressions and voice. The system needs to use an AI model to automatically analyze traffic conditions, weather risks, and user emotions, dynamically adjust traffic signal durations, and generate personalized notification texts and autonomous driving instructions based on user stress levels.

[0221] use Figure 14 The processing flow is explained.

[0222] Step 1:

[0223] The terminal collects real-time traffic video data, weather data, and user facial expressions and voice data from intersections and roads.

[0224] Inputs: Video stream from the camera, temperature, humidity, and wind speed values ​​from the weather sensor, and audio and video data from the microphone and camera.

[0225] The terminal performs data formatting and preliminary anomaly filtering on the raw data, such as removing invalid frames and noisy segments.

[0226] Output: Structured and effective traffic status information, environmental information, individual identification information, and physiological information data packets.

[0227] Step 2:

[0228] The terminal sends the processed data packets to the information aggregation device via a high-speed communication network.

[0229] Input: Data packets collected and preprocessed by the terminal.

[0230] The terminal merges and marks data from different sources according to timestamps and spatial locations to achieve data synchronization.

[0231] Output: A collection of multimodal data with time and location labels.

[0232] Step 3:

[0233] The information aggregation device uploads the collected data to the server via high-speed channels such as 5G.

[0234] Input: The multimodal data set after terminal merging.

[0235] The information aggregation device performs batch verification of data, eliminating duplicate or missing data sets.

[0236] Output: High-quality, complete real-time multi-source data files.

[0237] Step 4:

[0238] After receiving the data, the server calls the AI ​​analysis engine to perform target detection, trajectory analysis, and license plate recognition on the traffic video data, to conduct road condition risk assessment on the meteorological data, and to perform emotion recognition on the user's facial expressions and voice data.

[0239] Input: High-quality, validated multimodal data files.

[0240] The server uses a deep learning model to process video frames and numerical data, automatically labeling information such as vehicles, pedestrians, violations, and abnormal weather, and then extracting audio and video features to determine the user's psychological state.

[0241] Outputs include: detailed traffic flow information, violation records, road risk warnings, and user sentiment analysis reports.

[0242] Step 5:

[0243] Based on comprehensive analysis results, the server dynamically generates control commands for signal equipment and autonomous driving equipment and sends them to the relevant equipment.

[0244] Inputs: Traffic conditions, environmental alerts, and user psychological state analysis reports.

[0245] The server adjusts traffic light durations, switches emergency modes, and sets priority passage based on AI results, and also adjusts the speed, route, or driving mode of autonomous vehicles.

[0246] Output: Control signal command stream.

[0247] Step 6:

[0248] The server customizes personalized notification content based on the user's real-time mood and external conditions, and pushes it to the user through mobile terminals, in-vehicle systems, or voice broadcasts.

[0249] Input: User individual identification and psychological state, traffic and environmental comprehensive analysis.

[0250] The server combines the analysis results to generate prompts, such as estimated congestion time, severe weather warnings, and psychological counseling suggestions, and proactively notifies users in an appropriate manner.

[0251] Output: Personalized notification content and push history.

[0252] Step 7:

[0253] Users receive personalized notifications and adjust their behavior accordingly, such as choosing alternative routes or remaining calm.

[0254] Input: Notification information pushed by the server.

[0255] Users can provide feedback on notifications via their handheld devices (such as confirmations, suggestions, complaints, etc.). The server learns from this feedback and optimizes accordingly to improve the accuracy and user-friendliness of subsequent decisions.

[0256] Output: User feedback data, system self-optimization records.

[0257] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires sound representing user input regarding the result of the specific processing. The control unit 46A sends the sound data representing user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0258] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0259] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0260] For example, the collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart device 14 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0261] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart device 14.

[0262] Second Implementation Method

[0263] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0264] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. A server can be cited as an example of the data processing device 12.

[0265] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0266] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and communication I / F 44 are also connected to the bus 52.

[0267] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0268] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0269] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0270] Figure 4 This illustrates an example of the main functions of the data processing device 12 and the smart glasses 214. For example... Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0271] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0272] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0273] In the smart glasses 214, the processor 46 performs reception and output processing. The memory 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the memory 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is implemented by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48. Furthermore, the smart glasses 214 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290.

[0274] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0275] Example 1

[0276] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0277] Application Example 1

[0278] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0279] Example 2

[0280] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0281] Application Example 2

[0282] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0283] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A outputs the result of the specific processing to the speaker 240. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0284] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0285] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0286] For example, the collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart glasses 214 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0287] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart glasses 214.

[0288] Third Implementation Method

[0289] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0290] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. A server can be cited as an example of the data processing device 12.

[0291] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0292] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, display 343, and communication I / F 44 are also connected to the bus 52.

[0293] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0294] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0295] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0296] Figure 6 This illustrates an example of the main functions of the data processing device 12 and the head-mounted terminal 314. For example... Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0297] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0298] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0299] In the head-mounted terminal 314, the processor 46 performs the acceptance / output processing. The memory 50 stores the acceptance / output program 60. The processor 46 reads the acceptance / output program 60 from the memory 50 and executes the read acceptance / output program 60 on the RAM 48. The acceptance / output processing is implemented by the processor 46 operating as a control unit 46A according to the acceptance / output program 60 executed on the RAM 48.

[0300] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the head-mounted terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the head-mounted terminal 314 will be referred to as the "terminal".

[0301] Example 1

[0302] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0303] Application Example 1

[0304] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0305] Example 2

[0306] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0307] Application Example 2

[0308] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0309] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A outputs the result of the specific processing to the speaker 240 and the display 343. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0310] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0311] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0312] For example, the collection unit is implemented by the control unit 46A of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the head-mounted terminal 314 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 to analyze the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 to generate a menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12 to provide the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0313] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the head-mounted terminal 314.

[0314] Fourth Implementation Method

[0315] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0316] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. A server can be cited as an example of the data processing device 12.

[0317] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0318] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, controlled object 443, and communication I / F 44 are also connected to the bus 52.

[0319] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0320] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to photograph the area around robot 414 (e.g., the field of view defined by a perspective equivalent to the field of vision of an average healthy person).

[0321] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0322] The controlled object 443 includes a display device, LEDs (light-emitting diodes) for the eyes, and motors for driving the arms, hands, and feet. The posture or movement of the robot 414 is controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0323] Figure 8 This illustrates an example of the main functions of the data processing device 12 and the robot 414. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0324] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0325] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0326] In robot 414, the processor 46 performs the acceptance and output processing. The memory 50 stores the acceptance and output program 60. The processor 46 reads the acceptance and output program 60 from the memory 50 and executes the read acceptance and output program 60 on RAM 48. The acceptance and output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance and output program 60 executed on RAM 48.

[0327] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal".

[0328] Example 1

[0329] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0330] Application Example 1

[0331] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0332] Example 2

[0333] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0334] Application Example 2

[0335] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0336] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the controlled object 443. The microphone 238 acquires sound input representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0337] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0338] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0339] For example, the collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the robot 414 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the robot 414 and the control object 443 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0340] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the robot 414.

[0341] Furthermore, the emotion-specific model 59, acting as an emotion engine, can determine a user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine a user's emotion based on an emotion graph that serves as a specific mapping (see [reference]). Figure 9 The system determines the user's emotions. Furthermore, the emotion-specific model 59 can similarly determine the robot's emotions, and the specific processing unit 290 performs specific processing based on the robot's emotions.

[0342] Figure 9 This is a diagram representing an emotion map 400 that maps multiple emotions. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotion is. On the outer side of the concentric circles, emotions representing states or behaviors arising from mood are arranged. Emotions are concepts that include feelings and mental states. Emotions generated by reactions occurring in the brain are arranged roughly to the left of the concentric circles. Emotions derived from situational judgments are arranged roughly to the right of the concentric circles. Emotions generated by reactions occurring in the brain and derived from situational judgments are arranged roughly above and below the concentric circles. Furthermore, "pleasant" emotions are arranged above the concentric circles, and "unpleasant" emotions are arranged below them. Thus, in the emotion map 400, multiple emotions are mapped based on the structure that generates emotions, and emotions that are likely to occur simultaneously are mapped close to each other.

[0343] These emotions are distributed at the three o'clock position of the emotion map 400, typically fluctuating between peace and anxiety. In the right half of the emotion map 400, situational awareness dominates over internal sensation, thus resulting in an impression of calm.

[0344] The inner side of the emotion map 400 represents the inner state, while the outer side represents behavior. Therefore, the further outward you are from the emotion map 400, the more visible the emotion becomes (manifested in behavior).

[0345] Here, human emotions are based on various balances such as posture and blood sugar levels. When these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotions in robots, cars, motorcycles, etc., can also be created in the following way: based on various balances such as posture and remaining battery power, when these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Voice Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the sensory-dominated region, called "response," are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the situational cognition-dominated region, called "situation," are arranged.

[0346] In the emotion map, two types of emotions that promote learning are defined. One is a negative emotion on the situational side, in the middle or peripheral region of "repentance" or "reflection." This occurs when the robot experiences negative emotions such as "I don't want to experience this feeling again" or "I don't want to be blamed again." The other is a positive emotion on the response side, near the "desire" region. This occurs when there are positive feelings such as "wanting more" or "wanting to know more."

[0347] The emotion-specific model 59 inputs user input into a pre-trained neural network to obtain emotion values ​​representing each emotion shown in the emotion map 400, thereby determining the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network... Figure 10 As shown in the sentiment graph 900, it was trained in a way that sentiments that are configured close to each other have similar values. Figure 10 The text shows examples of emotions such as "peace of mind", "stability", and "reassurance" that have similar emotion values.

[0348] The above description focuses on the functions of the data processing device 12, but the system of this disclosure is not necessarily installed on a server. The system of this disclosure can also be installed as a general information processing system. This disclosure can also be installed, for example, as a software program running on a personal computer, an application running on a smartphone, etc. The method of this disclosure can also be provided to users in the form of SaaS (Software as a Service).

[0349] In the above embodiments, an example of a specific process being performed by a single computer 22 is given. However, the technology disclosed herein is not limited to this, and the specific process can also be distributed among multiple computers, including computer 22. For example, the data generation model 58 can be located on an external device of the data processing apparatus 12, where data is generated based on the input data.

[0350] In the above embodiments, examples of storing a specific processing program 56 in the memory 32 have been described, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0351] Alternatively, a specific processing program 56 may be pre-stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 according to the requirements of the data processing device 12.

[0352] In addition, it is not necessary to store all the specific processing program 56 in the storage device such as the server connected to the data processing device 12 via the network 54 or in the memory 32; a portion of the specific processing program 56 may be stored in advance.

[0353] As hardware resources for performing specific processes, various processors, as shown below, can be used. For example, a CPU can be listed as a processor, which functions as a general-purpose processor that performs specific processes by executing software, i.e., a program. Furthermore, processors can be listed as special-purpose circuits such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application-Specific Integrated Circuits), which are processors with circuitry specifically designed to perform specific processes. Each processor has built-in or connected memory, and each processor executes specific processes using that memory.

[0354] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resources for performing a specific process can be a single processor.

[0355] As an example of a single processor, there are two approaches: First, a processor is composed of a combination of one or more CPUs and software, which functions as a hardware resource to perform a specific process; second, as represented by a SoC (System-on-a-chip), a processor is used to implement the functionality of the entire system, which includes multiple hardware resources for performing a specific process, using a single IC (Integrated Circuit) chip. In this way, the specific process is implemented by using one or more of the aforementioned processors as hardware resources.

[0356] Furthermore, the hardware architecture of these various processors, more specifically, can utilize circuits that combine semiconductor elements and other circuit components. Moreover, the specific process described above is just one example. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps added, or the processing order changed.

[0357] The descriptions and illustrations above are detailed explanations of a portion of the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the above descriptions of the structure, function, effect, and results are just one example of the structure, function, effect, and results of a portion of the technology disclosed herein. Therefore, without departing from the spirit of the technology disclosed herein, unnecessary parts may be deleted, new elements added, or replacements may be made to the descriptions and illustrations above. Furthermore, to avoid confusion and facilitate understanding of a portion of the technology disclosed herein, explanations of common technical knowledge that do not require special explanation under the premise of being able to implement the technology disclosed herein have been omitted from the descriptions and illustrations above.

[0358] All documents, patent applications and technical specifications set forth in this specification are incorporated herein by reference to the same extent that each document, patent application and technical specification is specifically and individually described therein and referenced by reference.

[0359] In addition, the following notes are provided in response to the above explanation.

[0360] Example 1

[0361] (Note 1)

[0362] An information processing system includes: a communication device disposed on an information display device for transmitting and receiving data; a data acquisition device consisting of an imaging device for acquiring vehicle dynamic information and a sensor device for acquiring environmental information; a data consolidation device for receiving data via a high-speed communication network and integrating and encapsulating it in chronological order; a data processing device for receiving various types of information from the data consolidation device and preprocessing and standardizing the information; an analysis device deployed on the information processing device for automatically inferring vehicle identification, illegal behavior detection, traffic flow, accidents, and road condition risks using generative artificial intelligence models and optical character recognition technology; a control device for dynamically controlling the display status of a signal display device based on the analysis results of the analysis device; and a management support device for visualizing traffic data and analysis results and enabling remote operation or management.

[0363] (Note 2)

[0364] According to the information processing system described in Appendix 1, the parsing device includes image analysis and time series analysis using a generative artificial intelligence model to simultaneously achieve vehicle attribute identification, traffic violation identification, traffic flow estimation, traffic accident detection, and road icing or snow accumulation risk prediction.

[0365] (Note 3)

[0366] According to the information processing system described in Appendix 1, the control device optimizes the working mode and display interval of the signal display device in real time based on the output of the analysis device, thereby realizing dynamic response processing such as congestion relief, accident response, and emergency vehicle priority passage route setting.

[0367] Application Example 1

[0368] (Note 1)

[0369] An information processing system includes: an information communication device located at a signal display device; a sensing device for collecting physical phenomenon data; a sensing device for collecting environmental information data; an information aggregation device for aggregating data from the sensing devices via a broadband information communication network; an information processing device for parsing data sent by the information aggregation device; a control device for controlling the signal display device based on the parsing results of the information processing device; an information processing device for inferring the user's state based on the user's biometric and voice information; and an output processing device for automatically generating notification information based on the parsing and inference results using a generative artificial intelligence model.

[0370] (Note 2)

[0371] According to the information processing system described in Appendix 1, the information processing device includes a device for the flow rate of specific objects and the occurrence of events, a device for identifying water bodies that violate rules, a device for predicting dangerous states based on physical phenomena and environmental information, a device for analyzing biological and voice information and estimating user states, a device for optimizing notification content and signal display states by combining the analysis results and the estimation results, and a device for generating prompt statements for generative artificial intelligence models.

[0372] (Note 3)

[0373] According to the information processing system described in Appendix 1, the control device has means for dynamically and autonomously changing the display state of the signal display device to optimize the flow of objects and respond to accidents, and includes means for controlling the warning output and notification method according to the user's presumed state.

[0374] Example 2

[0375] (Note 1)

[0376] An information processing system includes: an information communication infrastructure mounted on a remote control device; an information acquisition device for acquiring dynamic image information of vehicles and pedestrians; an information acquisition device for acquiring environmental status information; an information aggregation device for integrating the information acquired by the information acquisition device through a high-bandwidth communication line; an information analysis device for analyzing the information acquired by the information aggregation device; a control device for controlling the remote operation device based on the analysis results of the information analysis device; an information analysis device for identifying individual identifiers and violation status; an information analysis device for predicting surface changes based on environmental status information; an information analysis device for analyzing user image posture information and voice information and determining psychological state; and an information processing device for adjusting notification methods or control strategies using the psychological state analysis results.

[0377] (Note 2)

[0378] According to the information processing system described in Appendix 1, the information analysis device performs dynamic measurement, violation status identification, surface change prediction, and psychological state analysis.

[0379] (Note 3)

[0380] According to the information processing system described in Appendix 1, the control device dynamically adjusts the actions and notification content of the remote operation device based on dynamic information, environmental information, and the user's psychological state.

[0381] Application Example 2

[0382] (Note 1)

[0383] An information processing system includes: a device for collecting traffic state information and environmental information via a communication medium located on a signaling device; an observation device for collecting traffic state information; an observation device for collecting environmental information; an observation device for collecting individual identification information and physiological information; an information aggregation device for aggregating data output from the observation devices via a high-speed communication path; a computational processing device for analyzing data from the information aggregation device; a control device for generating control signals based on the analysis results of the computational processing device and the individual's physiological information, and controlling the signaling device and autonomous driving equipment; and a notification output device for providing personalized information output based on the analysis results and the individual's physiological information.

[0384] (Note 2)

[0385] According to the information processing system described in Appendix 1, the computing device has the functions of making inferences based on traffic flow and obstacle occurrence information, identifying entities that violate regulations, predicting road condition risks based on environmental conditions, and inferring individual psychological states based on physiological information.

[0386] (Note 3)

[0387] According to the information processing system described in Appendix 1, the control device has the functions of automatically performing traffic congestion relief, responding to obstacles, and controlling actions based on individual psychological states.

Claims

1. An information processing system, characterized in that, include: Communication infrastructure installed on traffic lights; Data acquisition device used for collecting traffic video data; Data acquisition devices used for collecting meteorological data; A data aggregation device for aggregating data acquired by the data acquisition device via a high-speed communication network; A parsing device for parsing the data sent by the data aggregation device; and A control device for controlling traffic lights based on the analysis results of the analysis device.

2. The information processing system according to claim 1, characterized in that, The analysis device includes a device for identifying traffic flow and traffic accidents, a device for identifying vehicles violating traffic laws, and a device for predicting road icing and snow accumulation.

3. The information processing system according to claim 1, characterized in that, The control device includes a means for dynamically changing the display status of traffic lights to alleviate traffic congestion and respond to traffic accidents.

Images