System

Abstract

A system is provided.SOLUTION: A system comprising: means for detecting an abnormal behavior by analyzing video data using a generation model; means for generating a voice announcement using the generation model and notifying the inside of a store of the voice announcement; means for notifying a staff member of an alert when the abnormal behavior is detected; and means for performing on-site check based on the alert notified to the staff member.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] This invention relates to a security system for efficiently preventing both shoplifting in offline stores and illegal online resale. Conventional systems do not centrally manage in-store shoplifting prevention and online resale detection, requiring separate countermeasures and resulting in a lack of efficiency. Furthermore, the systems require excessive effort from store staff and online platform administrators, making it necessary to reduce the workload. [Means for solving the problem]

[0005] The present invention provides a system that detects abnormal behavior through video data analysis using a generative model and includes a means for notifying store staff via audio announcements, a means for alerting staff when abnormal behavior is detected, and a means for staff to conduct on-site inspections. The system also includes a means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, a means for sending warning messages to sellers determined to be fraudulent, and a means for suspending listings determined to be fraudulent. This allows for centralized security measures both offline and online, thereby enabling efficient prevention of shoplifting and fraudulent resale.

[0006] A "generative model" is a type of artificial intelligence trained to perform pattern recognition and predictions based on data.

[0007] "Video data" means visual information captured by a camera or other image capture device.

[0008] "Abnormal behavior" refers to behavior that is judged to deviate from normal patterns of behavior, and in the context of crime prevention in particular, it often indicates illegal activity such as shoplifting.

[0009] "Voice announcements" are automatically generated voice messages that are broadcast within the store via speakers.

[0010] "Staff" refers to employees working at a store, and are responsible for various tasks, including security.

[0011] An "alert" is a warning given when an abnormal event occurs, and may be given by voice, visual, or data communication.

[0012] "On-site inspection" refers to the act of staff actually going to a designated area and checking the local situation.

[0013] "Listing Information" means information relating to products and services offered for sale on the Online Platforms.

[0014] A "pattern recognition algorithm" is a computational method used to identify specific patterns or trends in data.

[0015] "Fraudulent listing" means the listing of goods or services that deviates from normal market prices or sales methods and is deemed to be illegal or commercially improper.

[0016] A "warning message" is a notification sent to relevant parties when the system detects an abnormality or fraudulent activity.

[0017] A "pause" is a procedure that temporarily stops a particular action or behavior until a problem is resolved.

[0018] A "system" is a collection of multiple elements that work together to perform a specific function. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The security system of the present invention employs technology that uses generative models to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[0041] In-store monitoring and announcements

[0042] Video data collection and analysis

[0043] server

[0044] The server collects video data in real time from multiple cameras in the store. The video data received from the cameras is analyzed using image recognition algorithms. For example, trained models are used to detect specific behavioral patterns (e.g., putting items into a bag or attempting to take away a large number of items at once).

[0045] Alert generation and notification

[0046] server

[0047] When the server detects abnormal behavior, it generates an alert based on that information, which is then sent to the store staff's terminal.

[0048] Terminal

[0049] The store staff's devices receive the alert sent from the server and display a notification, such as "Suspicious activity has been detected on shelf number 2."

[0050] Real-time announcements

[0051] server

[0052] The server uses the generative model to automatically generate voice announcements and broadcast them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation."

[0053] User (store staff)

[0054] Staff can check the alert on the terminal and quickly move to the relevant area to check the situation. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[0055] Online resale monitoring

[0056] Collection and analysis of listing information

[0057] server

[0058] The server automatically collects listings from online platforms, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[0059] Fake listing detection and notification

[0060] server

[0061] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[0062] Terminal

[0063] The administrator's device receives the flag notification from the server and displays the notification. For example, the notification may say, "A particular listing may be fraudulent."

[0064] Warning messages and pauses

[0065] User (Administrator)

[0066] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, they can send a message saying, "This listing is being temporarily suspended due to the possibility of fraudulent transactions." If necessary, they can temporarily suspend the listing and conduct further investigation.

[0067] Specific examples

[0068] Specific examples of in-store monitoring and announcements

[0069] 1. The server analyzes camera footage in the store and detects suspicious behavior, such as a customer putting a large number of items into a bag at once.

[0070] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[0071] 3. Staff check the alert on their device, go to the relevant area and check the situation on site.

[0072] 4. At the same time, the server automatically generates an audio announcement and plays it throughout the store to raise awareness of crime prevention.

[0073] Examples of online resale monitoring

[0074] 1. The server collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that a specific brand of expensive watch has been repeatedly listed at abnormally low prices.

[0075] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[0076] 3. The administrator checks the notification on the device and sends a warning message to the seller stating, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0077] 4. The administrator will suspend the relevant listing and conduct further detailed checks and investigations.

[0078] The above is an example of a system of the present invention that can effectively prevent fraud both in-store and online.

[0079] The processing flow will be explained below.

[0080] In-store monitoring and announcement processing flow

[0081] Step 1:

[0082] server

[0083] The server collects video data in real time from cameras installed in the store, and the video data is immediately sent to an image recognition algorithm.

[0084] Step 2:

[0085] server

[0086] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large amounts of merchandise).

[0087] Step 3:

[0088] server

[0089] If any abnormal behavior is detected, the server immediately generates an alert, which includes the specific details and location of the detected behavior.

[0090] Step 4:

[0091] server

[0092] The server then sends the generated alert to the store staff's device, specifically including a notification such as "Suspicious activity has been detected on shelf number 2."

[0093] Step 5:

[0094] Terminal

[0095] Store staff's devices receive and display alerts from the server, allowing them to quickly identify the location of any suspicious activity.

[0096] Step 6:

[0097] User (store staff)

[0098] Staff check the alerts received on their devices, rush to the scene, confirm the actual situation, and take appropriate action.

[0099] Step 7:

[0100] server

[0101] The server uses the generative model to automatically generate voice announcements when abnormal behavior is detected, such as "Customers, please be aware that security cameras are currently in operation."

[0102] Step 8:

[0103] server

[0104] The generated audio announcements are played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[0105] Online Resale Monitoring Process Flow

[0106] Step 1:

[0107] server

[0108] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[0109] Step 2:

[0110] server

[0111] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[0112] Step 3:

[0113] server

[0114] If a listing is determined to be fraudulent, the server will flag the listing, which will include details about the listing and any anomalies.

[0115] Step 4:

[0116] server

[0117] The server then sends the generated flag to the terminal of the online platform administrator, which contains a notification such as "A particular listing may be fraudulent."

[0118] Step 5:

[0119] Terminal

[0120] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[0121] Step 6:

[0122] User (Administrator)

[0123] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0124] Step 7:

[0125] User (Administrator)

[0126] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[0127] In this way, the system of the present invention can efficiently implement security measures both offline and online.

[0128] Example 1

[0129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0130] Preventing fraudulent activities is a major challenge in modern commercial facilities. Detecting and responding to shoplifting and fraudulent resale activities in real time is particularly challenging. Addressing these challenges requires building an efficient and accurate surveillance system. However, conventional systems have struggled to quickly and effectively detect and respond to these fraudulent activities. Therefore, the present invention aims to provide a new system that utilizes generative models to detect fraudulent activities in real time and respond quickly to them.

[0131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0132] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the facility, and means for notifying workers of an alert when abnormal behavior is detected. This makes it possible to detect abnormal behavior such as shoplifting in real time and prevent fraudulent activity through audio announcements. Furthermore, the server includes means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, means for sending warning messages to sellers determined to be fraudulent, and means for suspending listings determined to be fraudulent, thereby making it possible to effectively monitor and prevent fraudulent online resale activities.

[0133] A "generative model" is a model that uses machine learning algorithms to generate information or patterns from data.

[0134] "Video data" refers to data containing visual information obtained from a camera, video recording device, or the like.

[0135] "Abnormal behavior" is behavior that deviates from normal patterns of behavior and indicates misconduct or behavior that requires attention.

[0136] A "voice announcement" is a voice message automatically generated using voice synthesis technology.

[0137] An "alert" is a warning notification sent when the system detects an abnormality.

[0138] "Worker" refers to a person in charge of monitoring and responding within a store or facility.

[0139] "Listing Information" means the information provided when a product is offered for sale on an online platform.

[0140] A "pattern recognition algorithm" is an algorithm that identifies specific patterns or features in data.

[0141] A "warning message" is a message sent to notify of irregularities or abnormalities.

[0142] "Pause" is the act of temporarily halting a particular action or process.

[0143] The system of the present invention employs techniques that leverage generative models and pattern recognition algorithms to effectively monitor and prevent fraud in-store and online.

[0144] In-store monitoring and announcements

[0145] Video data collection and analysis

[0146] The server collects video data in real time from cameras installed in the store. The cameras are installed on the ceiling and shelves, and the viewing angle and resolution are set according to the store's layout. The collected video data is analyzed using image recognition algorithms such as the YOLO model using TensorFlow. Specifically, suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once) are detected.

[0147] Alert generation and notification

[0148] When the server detects suspicious behavior, it generates an alert and sends the information in JSON format to the store staff's device. The transmission is carried out using a REST API using the HTTP protocol. The device displays the received alert as a pop-up notification, for example, saying, "Suspicious behavior has been detected on shelf number 2."

[0149] Real-time announcements

[0150] The server automatically generates a voice announcement using a generative model (e.g., Google Text-to-Speech API) and broadcasts it throughout the store. The announcement includes a warning message such as, "Security cameras are in operation, please be careful." The user (store staff) checks the alert and quickly moves to the relevant area to check the situation.

[0151] Online resale monitoring

[0152] Collection and analysis of listing information

[0153] The server periodically collects listing information from major online platforms using scraping technology. The collected information is then analyzed using pattern recognition algorithms, such as logistic regression models using scikit-learn, to identify fraudulent listings. For example, pricing that deviates significantly from normal market prices or a large number of listings in a short period of time are monitored.

[0154] Fake listing detection and notification

[0155] The server generates a "fraudulent" flag for listings that are determined to be fraudulent and sends that information in JSON format to the online platform administrator's device. The notification includes a message such as "A specific listing may be fraudulent." The device then displays the received flag as a pop-up notification.

[0156] Sending warning messages and suspending listings

[0157] The user (administrator) checks the notification and sends a warning message to the relevant seller, saying, "This listing is temporarily suspended due to the possibility of fraudulent transactions." The user also temporarily suspends the listing through the system interface and conducts additional investigation.

[0158] Specific examples

[0159] Specific examples of in-store monitoring and announcements

[0160] 1. The server analyzes camera footage in the store and detects customers putting large amounts of items into bags at once.

[0161] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[0162] 3. The user (store staff member) checks the alert on the device, goes to the relevant area and checks the situation on site.

[0163] 4. The server automatically generates voice announcements and plays them throughout the store to raise awareness of crime prevention.

[0164] Examples of online resale monitoring

[0165] 1. The server collects listing information from e-commerce sites and uses a pattern recognition algorithm to detect when expensive watches from a particular brand are repeatedly listed at abnormally low prices.

[0166] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[0167] 3. The user (administrator) checks the notification on their device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0168] 4. The user (administrator) will suspend the relevant listing and conduct additional detailed checks and investigations.

[0169] The above is an embodiment of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in-store and online.

[0170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0171] In-store monitoring and announcements

[0172] Step 1: Collect video data

[0173] The server collects video data in real time from cameras installed in the store, such as those on the ceiling or on shelves, with the viewing angle and resolution set according to the store's layout.

[0174] Input: Video data from the camera

[0175] Output: Raw video data sent to the server

[0176] Step 2: Analyzing the video data

[0177] The server analyzes the collected video data using image recognition algorithms such as the YOLO model powered by TensorFlow, and specifically detects suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once).

[0178] Input: Raw video data stored on the server

[0179] Output: Analyzed video data and suspicious behavior detection results

[0180] Step 3: Generate an alert

[0181] If the server detects any suspicious activity as a result of the analysis, it generates an alert, which includes the camera's location and timestamp.

[0182] Input: Suspicious behavior detection result

[0183] Output: Alert information (location, timestamp, etc.)

[0184] Step 4: Alert Notification

[0185] The server sends the generated alert to the store staff's terminal using the REST API with the HTTP protocol. The transmission format is JSON. The terminal displays the received alert as a pop-up notification.

[0186] Input: Alert information

[0187] Output: Notification displayed on device (e.g. "Suspicious activity detected on shelf 2")

[0188] Step 5: Generate real-time announcements

[0189] The server uses a generative model (e.g., Google Text-to-Speech API) to generate audio announcements, such as "Security cameras are active, please be careful."

[0190] Input: Alert information

[0191] Output: Generated voice announcement data

[0192] Step 6: Sending real-time announcements

[0193] The server transmits the generated voice data to a speaker system in the store, and the announcement is broadcast throughout the store.

[0194] Input: Generated voice announcement data

[0195] Output: Announcement played in store

[0196] Step 7: Staff response

[0197] The user (store staff) checks the alert on the terminal, quickly moves to the relevant area, and checks the situation on the spot. If necessary, they can also play back and check the recorded data from the surveillance camera.

[0198] Input: Alert notification displayed on the terminal

[0199] Output: On-site confirmation and response

[0200] Online resale monitoring

[0201] Step 1: Gather your listing information

[0202] The server periodically collects listing information from major online platforms using scraping technology, such as the Python library Scrapy.

[0203] Input: Online platform webpage information

[0204] Output: Listing information data (product name, price, category, seller information, etc.)

[0205] Step 2: Listing Analysis

[0206] The server analyzes the collected listing information using pattern recognition algorithms such as logistic regression models using scikit-learn to identify fraudulent listings, such as those with prices that deviate significantly from normal market prices or those with a large number of listings in a short period of time.

[0207] Input: Listing information data

[0208] Output: Identification results of fraudulent listings (listing ID, price, reason for detection, etc.)

[0209] Step 3: Generate flags

[0210] Based on the analysis results, the server generates a "Fraud" flag for fraudulent listings. This flag contains information such as the seller ID, listing ID, and the reason for detection.

[0211] Input: Result of identifying fraudulent listings

[0212] Output: Invalid flag information

[0213] Step 4: Flag Notification

[0214] The server notifies the online platform administrator of the generated fraud flag via a REST API using the HTTP protocol. The transmission format is JSON. The terminal displays the received flag as a pop-up notification.

[0215] Input: Invalid flag information

[0216] Output: Notification displayed on device (e.g. "This particular listing may be fraudulent")

[0217] Step 5: Sending a warning message

[0218] The user (administrator) checks the notification and sends a warning message to the relevant seller, such as "This listing is temporarily suspended as it may be a fraudulent transaction."

[0219] Input: Invalid flag information

[0220] Output: Warning message sent to seller

[0221] Step 6: Pause your listing

[0222] The user (administrator) temporarily suspends the relevant listing through the system interface, and takes action to stop the listing from being published in order to conduct additional investigation.

[0223] Input: Invalid flag information

[0224] Output: Paused listings

[0225] Step 7: Conduct additional research

[0226] The user (administrator) checks the details of the suspected fraudulent listing and conducts further investigation as necessary, for example, by checking the seller's past transaction data or other listed items.

[0227] Enter: Paused listings

[0228] Output: Investigation results (determination of the suitability of the listing)

[0229] The above are the specific processing steps of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in stores and online.

[0230] (Application example 1)

[0231] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] Conventional security systems did not provide sufficient means to prevent shoplifting and unauthorized resales occurring in stores. This resulted in a heavy burden on staff and reduced security efficiency. Furthermore, manual monitoring and response was required for online unauthorized resales, making it difficult to detect fraudulent activity early. An efficient and effective solution to these issues was needed.

[0233] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0234] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the store, means for notifying staff of an alert when abnormal behavior is detected, means for conducting on-site inspections based on the alert notified to the staff, and means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, thereby enabling effective monitoring and prevention of fraudulent activities both in-store and online.

[0235] A "generative model" is a model that uses machine learning algorithms to automatically learn patterns from data and perform specific tasks.

[0236] "Video data" refers to image and video information collected using a camera.

[0237] "Abnormal behavior" refers to actions or behavior that deviate from pre-established normal behavior patterns.

[0238] A "pattern recognition algorithm" is an algorithm for identifying and classifying specific patterns in data.

[0239] "Voice announcements" refer to audio notifications and guidance that are automatically generated using generative models.

[0240] An "alert" refers to a warning or notification sent to staff when the system detects abnormal behavior.

[0241] "Staff" refers to employees working in the store and those in charge of monitoring.

[0242] "Listing information" refers to detailed information about products sold on online marketplaces and e-commerce sites.

[0243] "Fraudulent listings" refer to products that are sold at prices significantly different from market value or through abnormal sales patterns.

[0244] "Warning Message" means a message sent to a seller or staff member to alert them to possible fraudulent activity.

[0245] The present invention is a security system that aims to efficiently detect and prevent fraudulent activities in stores and online using a generative AI model. Specific embodiments are described below.

[0246] In-store surveillance system

[0247] The server collects video data in real time from multiple cameras in the store. This video data is analyzed using image recognition algorithms (e.g., OpenCV or Caffe models). A trained generative model is used to detect specific behavioral patterns (e.g., putting items into a bag or trying to take away a large number of items at once).

[0248] When the server detects abnormal behavior, it immediately generates an alert and sends a notification to the store staff's device (such as a smartphone or tablet). The staff's device displays specific information, such as "Suspicious behavior has been detected on shelf number 2." The staff member checks the notification and quickly moves to the area in question to check the situation.

[0249] Furthermore, the server automatically generates voice announcements and sends them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation." This voice announcement will raise security awareness and is expected to have a deterrent effect on crime.

[0250] Online Resale Monitoring System

[0251] The server automatically collects listing information from major online marketplaces. The collected listing information is then analyzed using pattern recognition algorithms, analyzing information such as listing pricing and number of listings to identify listings that deviate from normal market prices and sales practices.

[0252] If the server detects fraudulent listing activity based on the analysis results, it generates a flag for the activity and sends it to the online platform administrator's device. The administrator's device displays a notification such as, "A particular listing may be fraudulent." The administrator then checks the notification and sends a warning message to the problematic seller. For example, the message might say, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0253] If necessary, the administrators will suspend the listing and conduct further detailed checks and investigations, which will enable them to quickly prevent unauthorized resale activities on the online platform.

[0254] Hardware and software used

[0255] Hardware used: security cameras, servers, smartphones, tablets

[0256] Software used: OpenCV (image recognition library), Caffe (deep learning model), REST API, push notification, Requests (HTTP request library)

[0257] Specific examples

[0258] For example, if there is ongoing fraudulent purchases or unfairly low sales of a particular product, the system will send a warning message such as, "The pricing of this product is inappropriate. We will conduct a detailed investigation."

[0259] Prompt Sentence Examples

[0260] "It uses image recognition algorithms to detect abnormal behavior in security camera footage in real time."

[0261] "We collect listing information from online platforms, detect abnormal behavioral patterns and generate alerts."

[0262] The above is a specific embodiment for carrying out the present invention, which enhances security both in-store and online, and enables rapid detection and response to fraudulent activity.

[0263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0264] Step 1:

[0265] The server collects video data in real time from multiple cameras in the store. The video data sent from the cameras is input. The server stores the video data and prepares it for analysis.

[0266] Step 2:

[0267] The server analyzes the collected video data using image recognition algorithms (OpenCV and Caffe models). Specifically, the video data is preprocessed and input into a generative model. The generative model detects specific behavioral patterns (e.g., putting an item into a bag) and outputs the results.

[0268] Step 3:

[0269] If the server detects abnormal behavior based on the output of the generative model, it generates an alert. This alert information is output and sent to the staff's terminal. The alert contains specific information such as "Suspicious behavior has been detected on shelf number 2."

[0270] Step 4:

[0271] The terminal displays the alert received from the server. The staff member checks the notification and prepares to quickly head to the relevant area. Specifically, the staff member checks the alert content on the screen and heads to the scene.

[0272] Step 5:

[0273] When abnormal behavior is detected, the server automatically generates a voice announcement using the generative model. The generated voice announcement is output. For example, it could say, "Attention customers, please be aware that security cameras are in operation."

[0274] Step 6:

[0275] The server collects listing information from major online marketplaces, which serves as input, stores the listing information in a database, and prepares it for analysis.

[0276] Step 7:

[0277] The server analyzes the collected listing information using a pattern recognition algorithm, detecting fraudulent listings based on the listing price and number of listings, and outputs the results.

[0278] Step 8:

[0279] If the server detects fraudulent listing activity, it generates a flag and sends a notification to the online platform administrator's device, stating that "a particular listing may be fraudulent."

[0280] Step 9:

[0281] The terminal displays the flag notification received from the server. The administrator checks the notification and sends a warning message to the problematic seller. Specifically, the administrator sends a message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0282] Step 10:

[0283] The user (administrator) can suspend any listings that are deemed fraudulent as necessary, and conduct additional detailed checks and investigations. Specifically, the user selects the relevant listing from the system's administration screen and clicks the suspend button.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] The security system of the present invention employs technology that combines a generative model and an emotion engine to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[0286] In-store monitoring and announcements

[0287] Video data collection and analysis

[0288] server

[0289] The server collects video data in real time from multiple cameras installed in the store. The video data received from the cameras is analyzed using image recognition algorithms and an emotion engine. For example, it detects specific behavioral patterns (e.g., hiding items in a bag, attempting to steal a large amount of merchandise) and changes in facial expressions (anxiety, impatience, etc.).

[0290] Alert generation and notification

[0291] server

[0292] When the server detects abnormal behavior or changes in emotion, it generates an alert based on that information, which includes the specific content and location of the detected behavior or emotion.

[0293] Terminal

[0294] The store staff's device receives the alert sent from the server and displays a notification, such as "Suspicious behavior and an anxious user expression were detected on shelf number 2."

[0295] Real-time announcements

[0296] server

[0297] The server uses the generative model and emotion engine to automatically generate voice announcements and broadcast them throughout the store. The content of the announcement changes dynamically depending on the user's emotional state. For example, a message such as "Customers, please be aware that security cameras are currently in operation." may be generated.

[0298] User (store staff)

[0299] Staff check the alert on their device and rush to the scene, where they can confirm the actual situation and take appropriate action. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[0300] Online resale monitoring

[0301] Collection and analysis of listing information

[0302] server

[0303] The server periodically collects listings from the online platform, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[0304] Fake listing detection and notification

[0305] server

[0306] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[0307] Terminal

[0308] The administrator's device receives the flag notification from the server and displays it, allowing the administrator to quickly identify any fraudulent listings.

[0309] Warning messages and pauses

[0310] User (Administrator)

[0311] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, a message such as "The price setting for this item is inappropriate. We will conduct a detailed investigation" will be sent. If necessary, the listing will be suspended and additional investigation will be carried out.

[0312] Specific examples

[0313] Specific examples of in-store monitoring and announcements

[0314] 1. Server: Analyzes in-store camera footage and detects suspicious behavior and anxious expressions from users. For example, if a customer puts a large number of items into a bag at once and looks anxious.

[0315] 2. Server: Generates an alert and notifies the staff member's device that "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf 2."

[0316] 3. Terminal: Staff check the alert on the terminal and rush to the area to check the situation.

[0317] 4. Server: Using the generative model and emotion engine, an automatically generated voice announcement is played in the store, informing customers, "Customers, please be aware that security cameras are in operation."

[0318] Examples of online resale monitoring

[0319] 1. Server: Collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a specific brand have been listed repeatedly at abnormally low prices.

[0320] 2. Server: Flags the listing as fraudulent and notifies the administrator's device.

[0321] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0322] 4. User (Administrator): Pause the listing and conduct additional detailed checks and investigations.

[0323] In this way, by combining emotion engines, the present invention achieves even more accurate fraud detection and response.

[0324] The processing flow will be explained below.

[0325] In-store monitoring and announcement processing flow

[0326] Step 1:

[0327] server

[0328] The server collects video data in real time from multiple cameras installed in the store, and the video data is immediately sent to the image recognition algorithm and emotion engine.

[0329] Step 2:

[0330] server

[0331] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[0332] Step 3:

[0333] server

[0334] The server uses an emotion engine to analyze the user's facial expressions, detecting, for example, changes in facial expression that indicate feelings of anxiety or impatience.

[0335] Step 4:

[0336] server

[0337] If any abnormal behavior or emotional changes are detected, the server immediately generates an alert, which includes the specific content and location of the detected behavior or emotion.

[0338] Step 5:

[0339] server

[0340] The server then sends the generated alert to the store staff's device, which contains a notification such as "Suspicious behavior and anxious user expression were detected on shelf number 2."

[0341] Step 6:

[0342] Terminal

[0343] Store staff's devices receive and display alerts from the server, allowing them to quickly grasp the location and circumstances of suspicious activity.

[0344] Step 7:

[0345] User (store staff)

[0346] Staff check the alerts received on their devices, rush to the scene, check the actual situation there, and take appropriate action if necessary.

[0347] Step 8:

[0348] server

[0349] The server uses the generative model and emotion engine to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[0350] Step 9:

[0351] server

[0352] The generated audio announcements are played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[0353] Online Resale Monitoring Process Flow

[0354] Step 1:

[0355] server

[0356] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[0357] Step 2:

[0358] server

[0359] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[0360] Step 3:

[0361] server

[0362] The server uses an emotion engine to analyze changes in emotions from sellers' profiles and comments, detecting, for example, when a seller appears unnaturally anxious.

[0363] Step 4:

[0364] server

[0365] If a listing is deemed fraudulent or emotionally abnormal, the server flags the listing, which includes details about the listing and the abnormality.

[0366] Step 5:

[0367] server

[0368] The server then sends the generated flag to the terminal of the online platform administrator, which contains a notification such as "A particular listing may be fraudulent."

[0369] Step 6:

[0370] Terminal

[0371] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[0372] Step 7:

[0373] User (Administrator)

[0374] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0375] Step 8:

[0376] User (Administrator)

[0377] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[0378] In this way, by combining the emotion engine, the system of the present invention can achieve even more accurate fraud detection and response.

[0379] Example 2

[0380] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Conventional security systems monitor in-store shoplifting and online fraudulent resale separately, making it impossible to manage both in an integrated manner. Furthermore, there is a need for more accurate monitoring and notification that takes into account the emotional state of the user, rather than simply detecting abnormal behavior through video analysis. Furthermore, more advanced analysis and rapid notification are required to detect fraudulent listings and respond appropriately.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0383] In this invention, the server includes means for analyzing video data using a generative model and detecting abnormal behavior and changes in emotion, means for generating audio announcements using the generative model and emotion engine and notifying the facility, and means for sending an alert to a staff member's terminal when abnormal behavior or changes in emotion are detected. This makes it possible to accurately detect fraudulent behavior within the facility and take appropriate action taking into account changes in emotion.

[0384] Additionally, for monitoring online resale, the system includes a means for collecting listing information and using pattern recognition algorithms to detect fraudulent listings, a means for sending warning messages to sellers determined to be fraudulent, and a means for suspending listings determined to be fraudulent, making it possible to quickly detect fraudulent online resale activities and take appropriate measures.

[0385] A "generative model" is an algorithm that uses machine learning and deep learning techniques to generate data such as text, images, and audio.

[0386] "Video Data" means real-time or recorded visual data collected by a camera or other image capture device.

[0387] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior, and includes behavior that is deemed suspicious for security purposes.

[0388] "Changes in emotions" refers to fluctuations in the psychological state that can be inferred from the facial expressions and attitudes of the person being observed.

[0389] An "emotion engine" is an algorithm that analyzes emotions from image and video data and identifies their state.

[0390] "Voice announcement" is an output means for generating a specific message as voice and announcing it through a speaker.

[0391] An "alert" is a notification issued when the system detects an abnormal condition or suspicious behavior, prompting a warning or action.

[0392] A "terminal" is an electronic device (such as a computer, tablet, or smartphone) that allows staff or managers to receive information and perform operations.

[0393] "Listing information" refers to data containing detailed information about products listed on online marketplaces and auction sites.

[0394] A "pattern recognition algorithm" is an algorithm that extracts regularities and patterns from data and analyzes anomalies and characteristics.

[0395] "Fraudulent listing" refers to the listing of a product that deviates from normal market prices and sales methods and violates standards and regulations.

[0396] A "warning message" is a notification sent to notify of fraudulent activity or an abnormal state, and includes content urging improvement.

[0397] "Suspension" refers to the temporary interruption or cessation of a system or operation.

[0398] The present invention is a security system for monitoring and preventing in-store and online fraud. The system uses generative models and emotion engines to detect and respond to in-store shoplifting and online fraudulent resale activities.

[0399] In-store monitoring and announcements

[0400] Server: Multiple cameras are installed in the store to collect video data in real time. Specifically, IP cameras are used to send video data to the server via RTSP (Real Time Streaming Protocol).

[0401] Server: The collected video data is analyzed in real time using image recognition algorithms (such as OpenCV or Amazon Rekognition) running on the cloud. An emotion engine such as Microsoft Azure's Emotion API is also used to analyze the user's facial expressions and detect abnormal behavior and changes in emotion. For example, the system can detect when a customer picks up an item from a shelf and puts it in a bag, or when an anxious expression appears.

[0402] Server: Generates alerts based on the detection results and sends them to store staff terminals. The alerts include the content and location of the detected behavior and changes in emotions.

[0403] Server: Creates the content of the voice announcement using a generative AI model (e.g., GPT-3). Dynamically changes the content of the announcement based on information from the emotion engine. For example, a message such as "Customers, please be aware that security cameras are in operation" is generated and announced through the in-store speaker system.

[0404] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and take action as necessary. For example, they ask the customer, "Excuse me, is there anything I can help you with?"

[0405] Online resale monitoring

[0406] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[0407] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when a high-priced product of a particular brand is repeatedly listed at an abnormally low price.

[0408] Server: Based on the analysis results, flags suspicious listings as fraudulent and notifies the administrator's device. The notification includes information about the product, price, seller, etc.

[0409] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the listing will be suspended and further investigation will be conducted.

[0410] User (Administrator): After sending a warning message, we will conduct additional detailed checks and investigations, such as checking past transaction history and other listings by the seller to verify whether there has been any ongoing fraudulent activity.

[0411] Examples of specific examples and prompts

[0412] Specific examples of in-store monitoring and announcements

[0413] 1. Server: Collects camera footage from within the store and detects suspicious behavior and anxious expressions of customers. Example: A customer picks up multiple items from a shelf and puts them in a bag.

[0414] 2. Server: Based on the detected results, an alert is generated stating, "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf number 2," and notified to the staff member's device.

[0415] 3. Terminal: Staff checks the alert on the terminal and rushes to the area to check the situation. Example: After staff arrives at the scene, they ask the customer, "Excuse me, is there anything I can help you with?"

[0416] 4. Server: Using the generative model and emotion engine, an audio announcement is played throughout the store saying, "Customers, please be aware that security cameras are in operation."

[0417] Examples of online resale monitoring

[0418] 1. Server: Collects listing information via the API of an e-commerce site and detects that expensive watches of a particular brand have been listed repeatedly at abnormally low prices.

[0419] 2. Server: Flags the listing as fraudulent and sends a notification to the administrator's device saying, "A specific brand of watch has been repeatedly listed at an abnormal price."

[0420] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0421] 4. User (Administrator): Pause the item in question and investigate past sales history and other listings to identify any abnormal patterns.

[0422] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0423] In-store monitoring and announcements

[0424] Step 1: Collect video data

[0425] Server: Collects video data in real time from multiple cameras installed in the store. Specifically, the IP cameras send video data to the server using RTSP (Real Time Streaming Protocol).

[0426] Input: Real-time video data from IP cameras

[0427] Output: Raw video data stored on a server

[0428] Step 2: Analyzing the video data

[0429] Server: Analyzes collected video data using an image recognition algorithm (e.g., OpenCV or Amazon Rekognition) running on the cloud. It also analyzes the user's facial expressions using an emotion engine (e.g., Microsoft Azure's Emotion API). Detects when a customer exhibits certain suspicious behavior (e.g., hiding a product in a bag) or changes in emotion (e.g., anxiety, impatience, etc.).

[0430] Input: Raw video data

[0431] Data processing / data calculation: Analyzes video data using image recognition algorithms and emotion engines to detect suspicious behavior and changes in emotions

[0432] Output: Analysis results (presence or absence of suspicious behavior and emotional changes, location information)

[0433] Step 3: Generate alerts and notifications

[0434] Server: Based on the analysis results, if suspicious behavior or changes in emotion are detected, an alert is generated. The alert includes the details of the detected behavior, location, and changes in emotion. The generated alert is sent to the store staff's terminal.

[0435] Input: Analysis results

[0436] Data processing / data calculation: Alert information generation

[0437] Output: Alert notification sent to store staff's terminal

[0438] Step 4: Real-time announcements

[0439] Server: Uses a generative model (e.g., GPT-3) to create the content of the voice announcement. Dynamically changes the content of the announcement based on information from the emotion engine. For example, it generates a message such as "Customers, please be careful as security cameras are currently in operation," and announces it through the store's speaker system.

[0440] Input: Alert information, emotion engine analysis results

[0441] Data processing / data calculation: voice announcement generation

[0442] Output: Voice announcement played through the in-store speaker system

[0443] Step 5: On-site inspection

[0444] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and, if necessary, take appropriate action against the customer. For example, they might ask, "Excuse me, is there anything I can help you with?"

[0445] Input: Alert notification

[0446] Data processing / data calculation: On-site confirmation and response

[0447] Output: Response to suspicious behavior

[0448] Online resale monitoring

[0449] Step 1: Gather your listing information

[0450] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[0451] Input: Online Marketplace Listings API

[0452] Output: Listing data saved on the server

[0453] Step 2: Listing Analysis

[0454] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when expensive products of a particular brand are repeatedly listed at abnormally low prices.

[0455] Input: Listing information data

[0456] Data processing / data calculation: Analyzes data using pattern recognition algorithms to identify fraudulent listings

[0457] Output: Analysis results (whether or not there is fraudulent listing, seller information)

[0458] Step 3: Detect and notify fraudulent listings

[0459] Server: Based on the analysis results, the listing is flagged as fraudulent and the information is sent to the administrator's device. The notification includes information about the product, price, seller, etc.

[0460] Input: Analysis results

[0461] Data processing / data calculation: Alert information generation

[0462] Output: Alert notification sent to the administrator's device

[0463] Step 4: Send warning messages and suspend listings

[0464] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the administrator can also suspend the relevant listing.

[0465] Input: Alert notification

[0466] Data processing / data calculation: generating warning messages and suspending listings

[0467] Output: Warning message sent to seller, listing suspension status

[0468] Step 5: Further investigation

[0469] User (Administrator): After sending a warning message to the seller, conduct additional detailed checks and investigations, such as checking past transaction history and other listings to verify whether there is ongoing fraudulent activity.

[0470] Input: Seller's transaction history, past listing information

[0471] Data processing / data calculation: Analysis of transaction history and listing information

[0472] Output: Report of findings and required actions

[0473] (Application example 2)

[0474] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0475] Preventing shoplifting and vandalism is crucial for modern stores, but many current security systems have difficulty quickly and accurately detecting suspicious behavior or changes in user emotion. Furthermore, fraudulent listings on online platforms are on the rise, requiring significant effort to monitor and respond to. To solve these problems, a security system that combines more advanced technologies is needed.

[0476] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0477] In this invention, the server includes a means for analyzing video data using a generative model to detect abnormal behavior and changes in facial expression, a means for generating audio announcements using the generative model and notifying the store, a means for notifying staff of an alert when abnormal behavior or changes in facial expression are detected, a means for conducting on-site inspections based on the alert notified to the staff, and a means for analyzing the user's emotional state using an emotion engine and generating an appropriate warning message. This makes it possible to quickly and accurately detect suspicious behavior and changes in facial expression and take appropriate action. It also makes it possible to detect fraudulent listings online and quickly respond to them.

[0478] A "generative model" is a type of machine learning algorithm that learns patterns from collected data and makes inferences based on new data.

[0479] "Video data" is a data format that refers to images and video information captured by cameras and other visual sensors.

[0480] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and indicates misconduct or risky behavior.

[0481] "Changes in facial expression" refers to changes in emotional state indicated by facial muscle movements, and is used to detect abnormal states such as anxiety or impatience.

[0482] "Voice announcement" is a means of conveying information through voice, and uses a generative model to automatically generate messages appropriate to the situation.

[0483] An "alert" refers to a warning message or notification issued when abnormal behavior or conditions are detected.

[0484] "Staff" refers to employees who are engaged in the operation of stores and facilities and who are responsible for safety management and customer service.

[0485] An "emotion engine" refers to technology that analyzes a user's facial expressions and behavioral data to determine their emotional state.

[0486] A "warning message" is a written or audio message that alerts a target or administrator when fraud or an abnormal condition is detected.

[0487] "Online platform" refers to a system or website for conducting transactions or exchanging information over the Internet.

[0488] "Fraudulent listing" refers to the act of selling products on online platforms with inappropriate pricing or false information.

[0489] A "pattern recognition algorithm" is a technology that automatically identifies specific patterns in data and is used to analyze image and text data.

[0490] "Listing information" refers to data on an online platform that lists product details, prices, etc.

[0491] The security system of the present invention uses a generative model and an emotion engine to monitor in-store security and fraudulent listings on online platforms. A specific system configuration and processing method for implementing the present invention are described in detail below.

[0492] System Configuration

[0493] Hardware Configuration

[0494] 1. Server:

[0495] A high-performance data analysis server (e.g., an Amazon EC2 instance).

[0496] 2. Camera:

[0497] High-resolution surveillance cameras (e.g. network cameras).

[0498] 3. Terminal:

[0499] An iOS or Android smartphone, or AR-enabled glasses (e.g., Google Glass).

[0500] Software Configuration

[0501] 1. Image Recognition Algorithm:

[0502] Uses OpenCV and TensorFlow.

[0503] 2. Emotion Engine:

[0504] It uses AWS's Amazon Rekognition and Microsoft Azure's Emotion API.

[0505] 3. Generative Model:

[0506] OpenAI's GPT-3.

[0507] Processing steps

[0508] In-store surveillance

[0509] 1. Camera footage collection and analysis:

[0510] Multiple cameras installed in the store transmit video in real time to a server, which then analyzes the video data using OpenCV and TensorFlow to detect specific behavioral patterns and changes in facial expressions.

[0511] 2. Alert generation and notification:

[0512] The server analyzes the video data using an emotion engine (AWS Rekognition) to determine the user's emotional state. If suspicious behavior or abnormal emotions are detected, an alert is generated and sent to the staff member's smartphone or smart glasses.

[0513] 3. Generate voice announcements:

[0514] If abnormal behavior or emotional changes are detected, the server uses OpenAI's GPT-3 to generate appropriate voice announcements, such as "Customers, please be aware that security cameras are in operation," which are broadcast throughout the store.

[0515] Online resale monitoring

[0516] 1. Listing information collection and analysis:

[0517] The server periodically collects listing information from online platforms and uses a pattern recognition algorithm to detect when a particular brand of high-priced product is repeatedly listed at an abnormally low price.

[0518] 2. Fake listing detection and notification:

[0519] The server flags the listing as fraudulent and notifies the administrator's terminal, who then checks the notification and sends a warning message to the seller.

[0520] Specific examples

[0521] 1. Camera footage analysis:

[0522] "The server receives video data from cameras in the store in real time. The received video data is analyzed using image recognition algorithms using OpenCV and TensorFlow. For example, it detects specific behavioral patterns (e.g., hiding products in a bag)."

[0523] 2. Sentiment analysis and alert generation:

[0524] "The received video data is analyzed using AWS's Amazon Rekognition to analyze the user's emotional state. If abnormal emotions such as anxiety or impatience are detected, the server generates an alert based on that information and sends it to the staff member's smartphone or smart glasses."

[0525] 3. Generate voice announcements:

[0526] "After detecting suspicious behavior or emotions, the server automatically generates an appropriate voice announcement using OpenAI's GPT-3. For example, a message such as, 'Customers, please be aware that security cameras are in operation, so please be careful,' is generated and broadcast throughout the store."

[0527] 4. Examples of prompts:

[0528] "Detect suspicious activity from the following camera footage and generate prompts to create alert messages corresponding to the activity:

[0529] Camera footage shows the customer carelessly putting a large amount of items into a bag and frequently looking back. What warning message corresponds to this behavior?

[0530] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0531] Step 1:

[0532] The server collects video data in real time from cameras installed in the store. The video data is transferred from the cameras to the server via a network. The video data is the input, and it becomes the basis for subsequent analysis processing.

[0533] Step 2:

[0534] The server analyzes the collected video data using OpenCV and TensorFlow. Specifically, it recognizes objects in the video and detects behavioral patterns. This allows it to identify abnormal behaviors and specific behavioral patterns. The output is data that includes abnormal behaviors and specific behaviors.

[0535] Step 3:

[0536] The server performs emotion analysis on the video data using AWS's Amazon Rekognition. It analyzes the facial expressions of people in the video and detects emotions such as anxiety or impatience. The input is the video data obtained in Step 2, and the output is the result of the emotion analysis.

[0537] Step 4:

[0538] If abnormal behavior or abnormal emotion is detected, the server generates an alert. The alert includes the content and location of the abnormal behavior or emotion. The alert generation includes a means to generate appropriate messages and warning content based on the analysis results. The output is alert notification data to staff.

[0539] Step 5:

[0540] The server sends the generated alert to the staff member's device (smartphone or smart glasses) via the network. The staff member's device receives the notification and displays it on its screen. The input is the generated alert data, and the output is the notification displayed on the staff member's device.

[0541] Step 6:

[0542] The server generates voice announcements using OpenAI's GPT-3. Based on the analysis of abnormal behavior and emotions, an appropriate warning message is automatically generated. The output is the generated voice announcement data.

[0543] Step 7:

[0544] The voice announcement data is broadcast through the sound system in the store. This allows everyone in the store to be alerted. The input is the voice announcement data generated in Step 6, and the output is the voice announcement from the sound system in the store.

[0545] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0548] [Second embodiment]

[0549] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0550] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0551] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0553] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0555] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0556] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0557] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0558] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0559] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0560] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0561] The security system of the present invention employs technology that uses generative models to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[0562] In-store monitoring and announcements

[0563] Video data collection and analysis

[0564] server

[0565] The server collects video data in real time from multiple cameras in the store. The video data received from the cameras is analyzed using image recognition algorithms. For example, trained models are used to detect specific behavioral patterns (e.g., putting items into a bag or attempting to take away a large number of items at once).

[0566] Alert generation and notification

[0567] server

[0568] When the server detects abnormal behavior, it generates an alert based on that information, which is then sent to the store staff's terminal.

[0569] Terminal

[0570] The store staff's devices receive the alert sent from the server and display a notification, such as "Suspicious activity has been detected on shelf number 2."

[0571] Real-time announcements

[0572] server

[0573] The server uses the generative model to automatically generate voice announcements and broadcast them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation."

[0574] User (store staff)

[0575] Staff can check the alert on the terminal and quickly move to the relevant area to check the situation. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[0576] Online resale monitoring

[0577] Collection and analysis of listing information

[0578] server

[0579] The server automatically collects listings from online platforms, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[0580] Fake listing detection and notification

[0581] server

[0582] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[0583] Terminal

[0584] The administrator's device receives the flag notification from the server and displays the notification, such as "A particular listing may be fraudulent."

[0585] Warning messages and pauses

[0586] User (Administrator)

[0587] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, they can send a message saying, "This listing is being temporarily suspended due to the possibility of fraudulent transactions." If necessary, they can temporarily suspend the listing and conduct additional investigations.

[0588] Specific examples

[0589] Specific examples of in-store monitoring and announcements

[0590] 1. The server analyzes camera footage in the store and detects suspicious behavior, such as a customer putting a large number of items into a bag at once.

[0591] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[0592] 3. Staff check the alert on their device, go to the relevant area and check the situation on site.

[0593] 4. At the same time, the server automatically generates an audio announcement and plays it throughout the store to raise awareness of crime prevention.

[0594] Examples of online resale monitoring

[0595] 1. The server collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a particular brand have been listed repeatedly at abnormally low prices.

[0596] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[0597] 3. The administrator checks the notification on the device and sends a warning message to the seller stating, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0598] 4. The administrator will suspend the relevant listing and conduct further detailed checks and investigations.

[0599] The above is an example of a system of the present invention that can effectively prevent fraud both in-store and online.

[0600] The processing flow will be explained below.

[0601] In-store monitoring and announcement processing flow

[0602] Step 1:

[0603] server

[0604] The server collects video data in real time from cameras installed in the store, and the video data is immediately sent to an image recognition algorithm.

[0605] Step 2:

[0606] server

[0607] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[0608] Step 3:

[0609] server

[0610] If any abnormal behavior is detected, the server immediately generates an alert, which includes the specific details and location of the detected behavior.

[0611] Step 4:

[0612] server

[0613] The server then sends the generated alert to the store staff's device, which contains a notification such as "Suspicious activity has been detected on shelf number 2."

[0614] Step 5:

[0615] Terminal

[0616] Store staff's devices receive and display alerts from the server, allowing them to quickly identify the location of any suspicious activity.

[0617] Step 6:

[0618] User (store staff)

[0619] Staff check the alerts received on their devices, rush to the scene, confirm the actual situation, and take appropriate action.

[0620] Step 7:

[0621] server

[0622] The server uses the generative model to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[0623] Step 8:

[0624] server

[0625] The generated audio announcement is played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[0626] Online Resale Monitoring Process Flow

[0627] Step 1:

[0628] server

[0629] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[0630] Step 2:

[0631] server

[0632] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[0633] Step 3:

[0634] server

[0635] If a listing is determined to be fraudulent, the server will flag the listing, which will include details about the listing and any anomalies.

[0636] Step 4:

[0637] server

[0638] The server then sends the generated flag to the online platform administrator's terminal, which contains a notification such as "A particular listing may be fraudulent."

[0639] Step 5:

[0640] Terminal

[0641] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[0642] Step 6:

[0643] User (Administrator)

[0644] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0645] Step 7:

[0646] User (Administrator)

[0647] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[0648] In this way, the system of the present invention can efficiently implement security measures both offline and online.

[0649] Example 1

[0650] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0651] Preventing fraudulent activities is a major challenge in modern commercial facilities. Detecting and responding to shoplifting and fraudulent resale activities in real time is particularly challenging. Addressing these challenges requires building an efficient and accurate surveillance system. However, conventional systems have struggled to quickly and effectively detect and respond to these fraudulent activities. Therefore, the present invention aims to provide a new system that utilizes generative models to detect fraudulent activities in real time and respond quickly to them.

[0652] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0653] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the facility, and means for notifying workers of an alert when abnormal behavior is detected. This makes it possible to detect abnormal behavior such as shoplifting in real time and prevent fraudulent activity through audio announcements. Furthermore, the server includes means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, means for sending warning messages to sellers determined to be fraudulent, and means for suspending listings determined to be fraudulent, thereby making it possible to effectively monitor and prevent fraudulent online resale activities.

[0654] A "generative model" is a model that uses machine learning algorithms to generate information or patterns from data.

[0655] "Video data" refers to data containing visual information obtained from a camera, video recording device, or the like.

[0656] "Abnormal behavior" is behavior that deviates from normal patterns of behavior and indicates misconduct or behavior that requires attention.

[0657] A "voice announcement" is a voice message automatically generated using voice synthesis technology.

[0658] An "alert" is a warning notification sent when the system detects an abnormality.

[0659] "Worker" refers to a person in charge of monitoring and responding within a store or facility.

[0660] "Listing Information" means the information provided when a product is offered for sale on an online platform.

[0661] A "pattern recognition algorithm" is an algorithm that identifies specific patterns or features in data.

[0662] A "warning message" is a message sent to notify of irregularities or abnormalities.

[0663] "Pause" is the act of temporarily halting a particular action or process.

[0664] The system of the present invention employs techniques that leverage generative models and pattern recognition algorithms to effectively monitor and prevent fraud in-store and online.

[0665] In-store monitoring and announcements

[0666] Video data collection and analysis

[0667] The server collects video data in real time from cameras installed in the store. The cameras are installed on the ceiling and shelves, and the viewing angle and resolution are set according to the store's layout. The collected video data is analyzed using image recognition algorithms such as the YOLO model using TensorFlow. Specifically, suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once) are detected.

[0668] Alert generation and notification

[0669] When the server detects suspicious behavior, it generates an alert and sends the information in JSON format to the store staff's device. The transmission is carried out using a REST API using the HTTP protocol. The device displays the received alert as a pop-up notification, for example, saying, "Suspicious behavior has been detected on shelf number 2."

[0670] Real-time announcements

[0671] The server automatically generates a voice announcement using a generative model (e.g., Google Text-to-Speech API) and broadcasts it throughout the store. The announcement includes a warning message such as, "Security cameras are in operation, please be careful." The user (store staff) checks the alert and quickly moves to the relevant area to check the situation.

[0672] Online resale monitoring

[0673] Collection and analysis of listing information

[0674] The server periodically collects listing information from major online platforms using scraping technology. The collected information is then analyzed using pattern recognition algorithms, such as logistic regression models using scikit-learn, to identify fraudulent listings. For example, pricing that deviates significantly from normal market prices or a large number of listings in a short period of time are monitored.

[0675] Fake listing detection and notification

[0676] The server generates a "fraudulent" flag for listings that are determined to be fraudulent and sends that information in JSON format to the online platform administrator's device. The notification includes a message such as "A specific listing may be fraudulent." The device then displays the received flag as a pop-up notification.

[0677] Sending warning messages and suspending listings

[0678] The user (administrator) checks the notification and sends a warning message to the relevant seller, saying, "This listing is temporarily suspended due to the possibility of fraudulent transactions." The user also temporarily suspends the listing through the system interface and conducts additional investigation.

[0679] Specific examples

[0680] Specific examples of in-store monitoring and announcements

[0681] 1. The server analyzes camera footage in the store and detects customers putting large amounts of items into bags at once.

[0682] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[0683] 3. The user (store staff member) checks the alert on the device, goes to the relevant area and checks the situation on site.

[0684] 4. The server automatically generates voice announcements and plays them throughout the store to raise awareness of crime prevention.

[0685] Examples of online resale monitoring

[0686] 1. The server collects listing information from e-commerce sites and uses a pattern recognition algorithm to detect when expensive watches from a particular brand are repeatedly listed at abnormally low prices.

[0687] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[0688] 3. The user (administrator) checks the notification on their device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0689] 4. The user (administrator) will suspend the relevant listing and conduct additional detailed checks and investigations.

[0690] The above is an embodiment of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in-store and online.

[0691] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0692] In-store monitoring and announcements

[0693] Step 1: Collect video data

[0694] The server collects video data in real time from cameras installed in the store, such as those on the ceiling or on shelves, with the viewing angle and resolution set according to the store's layout.

[0695] Input: Video data from the camera

[0696] Output: Raw video data sent to the server

[0697] Step 2: Analyzing the video data

[0698] The server analyzes the collected video data using image recognition algorithms such as the YOLO model powered by TensorFlow, and specifically detects suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once).

[0699] Input: Raw video data stored on the server

[0700] Output: Analyzed video data and suspicious behavior detection results

[0701] Step 3: Generate an alert

[0702] If the server detects any suspicious activity as a result of the analysis, it generates an alert, which includes the camera's location and timestamp.

[0703] Input: Suspicious behavior detection result

[0704] Output: Alert information (location, timestamp, etc.)

[0705] Step 4: Alert Notification

[0706] The server sends the generated alert to the store staff's terminal using the REST API with the HTTP protocol. The transmission format is JSON. The terminal displays the received alert as a pop-up notification.

[0707] Input: Alert information

[0708] Output: Notification displayed on device (e.g. "Suspicious activity detected on shelf 2")

[0709] Step 5: Generate real-time announcements

[0710] The server uses a generative model (e.g., Google Text-to-Speech API) to generate audio announcements, such as "Security cameras are active, please be careful."

[0711] Input: Alert information

[0712] Output: Generated voice announcement data

[0713] Step 6: Sending real-time announcements

[0714] The server transmits the generated voice data to a speaker system in the store, and the announcement is broadcast throughout the store.

[0715] Input: Generated voice announcement data

[0716] Output: Announcement played in store

[0717] Step 7: Staff response

[0718] The user (store staff) checks the alert on the terminal, quickly moves to the relevant area, and checks the situation on the spot. If necessary, they can also play back and check the recorded data from the surveillance camera.

[0719] Input: Alert notification displayed on the terminal

[0720] Output: On-site confirmation and response

[0721] Online resale monitoring

[0722] Step 1: Gather your listing information

[0723] The server periodically collects listing information from major online platforms using scraping technology, such as the Python library Scrapy.

[0724] Input: Online platform webpage information

[0725] Output: Listing information data (product name, price, category, seller information, etc.)

[0726] Step 2: Listing Analysis

[0727] The server analyzes the collected listing information using pattern recognition algorithms such as logistic regression models using scikit-learn to identify fraudulent listings, such as those with prices that deviate significantly from normal market prices or those with a large number of listings in a short period of time.

[0728] Input: Listing information data

[0729] Output: Identification results of fraudulent listings (listing ID, price, reason for detection, etc.)

[0730] Step 3: Generate flags

[0731] Based on the analysis results, the server generates a "Fraud" flag for fraudulent listings. This flag contains information such as the seller ID, listing ID, and the reason for detection.

[0732] Input: Result of identifying fraudulent listings

[0733] Output: Invalid flag information

[0734] Step 4: Flag Notification

[0735] The server notifies the online platform administrator of the generated fraud flag via a REST API using the HTTP protocol. The transmission format is JSON. The terminal displays the received flag as a pop-up notification.

[0736] Input: Invalid flag information

[0737] Output: Notification displayed on device (e.g. "This particular listing may be fraudulent")

[0738] Step 5: Sending a warning message

[0739] The user (administrator) checks the notification and sends a warning message to the relevant seller, such as "This listing is temporarily suspended as it may be a fraudulent transaction."

[0740] Input: Invalid flag information

[0741] Output: Warning message sent to seller

[0742] Step 6: Pause your listing

[0743] The user (administrator) temporarily suspends the relevant listing through the system interface, and takes action to stop the listing from being published in order to conduct additional investigation.

[0744] Input: Invalid flag information

[0745] Output: Paused listings

[0746] Step 7: Conduct additional research

[0747] The user (administrator) checks the details of the suspected fraudulent listing and conducts further investigation as necessary, for example, by checking the seller's past transaction data or other listed items.

[0748] Enter: Paused listings

[0749] Output: Investigation results (determination of the suitability of the listing)

[0750] The above are the specific processing steps of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in stores and online.

[0751] (Application example 1)

[0752] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] Conventional security systems did not provide sufficient means to prevent shoplifting and unauthorized resales occurring in stores. This resulted in a heavy burden on staff and reduced security efficiency. Furthermore, manual monitoring and response was required for online unauthorized resales, making it difficult to detect fraudulent activity early. An efficient and effective solution to these issues was needed.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0755] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the store, means for notifying staff of an alert when abnormal behavior is detected, means for conducting on-site inspections based on the alert notified to the staff, and means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, thereby enabling effective monitoring and prevention of fraudulent activities both in-store and online.

[0756] A "generative model" is a model that uses machine learning algorithms to automatically learn patterns from data and perform specific tasks.

[0757] "Video data" refers to image and video information collected using a camera.

[0758] "Abnormal behavior" refers to actions or behavior that deviate from pre-established normal behavior patterns.

[0759] A "pattern recognition algorithm" is an algorithm for identifying and classifying specific patterns in data.

[0760] "Voice announcements" refer to audio notifications and guidance that are automatically generated using generative models.

[0761] An "alert" refers to a warning or notification sent to staff when the system detects abnormal behavior.

[0762] "Staff" refers to employees working in the store and those in charge of monitoring.

[0763] "Listing information" refers to detailed information about products sold on online marketplaces and e-commerce sites.

[0764] "Fraudulent listings" refer to products that are sold at prices significantly different from market value or through abnormal sales patterns.

[0765] "Warning Message" means a message sent to a seller or staff member to alert them to possible fraudulent activity.

[0766] The present invention is a security system that aims to efficiently detect and prevent fraudulent activities in stores and online using a generative AI model. Specific embodiments are described below.

[0767] In-store surveillance system

[0768] The server collects video data in real time from multiple cameras in the store. This video data is analyzed using image recognition algorithms (e.g., OpenCV or Caffe models). A trained generative model is used to detect specific behavioral patterns (e.g., putting items into a bag or trying to take away a large number of items at once).

[0769] When the server detects abnormal behavior, it immediately generates an alert and sends a notification to the store staff's device (such as a smartphone or tablet). The staff's device displays specific information, such as "Suspicious behavior has been detected on shelf number 2." The staff member checks the notification and quickly moves to the area in question to check the situation.

[0770] Furthermore, the server automatically generates voice announcements and sends them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation." This voice announcement will raise security awareness and is expected to have a deterrent effect on crime.

[0771] Online Resale Monitoring System

[0772] The server automatically collects listing information from major online marketplaces. The collected listing information is then analyzed using pattern recognition algorithms, analyzing information such as listing pricing and number of listings to identify listings that deviate from normal market prices and sales practices.

[0773] If the server detects fraudulent listing activity based on the analysis results, it generates a flag for the activity and sends it to the online platform administrator's device. The administrator's device displays a notification such as, "A particular listing may be fraudulent." The administrator then checks the notification and sends a warning message to the problematic seller. For example, the message might say, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0774] If necessary, the administrators will suspend the listing and conduct further detailed checks and investigations, which will enable them to quickly prevent unauthorized resale activities on the online platform.

[0775] Hardware and software used

[0776] Hardware used: security cameras, servers, smartphones, tablets

[0777] Software used: OpenCV (image recognition library), Caffe (deep learning model), REST API, push notification, Requests (HTTP request library)

[0778] Specific examples

[0779] For example, if there is ongoing fraudulent purchases or unfairly low sales of a particular product, the system will send a warning message such as, "The pricing of this product is inappropriate. We will conduct a detailed investigation."

[0780] Prompt Sentence Examples

[0781] "It uses image recognition algorithms to detect abnormal behavior in security camera footage in real time."

[0782] "We collect listing information from online platforms, detect abnormal behavioral patterns and generate alerts."

[0783] The above is a specific embodiment for carrying out the present invention, which enhances security both in-store and online, and enables rapid detection and response to fraudulent activity.

[0784] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0785] Step 1:

[0786] The server collects video data in real time from multiple cameras in the store. The video data sent from the cameras is input. The server stores the video data and prepares it for analysis.

[0787] Step 2:

[0788] The server analyzes the collected video data using image recognition algorithms (OpenCV and Caffe models). Specifically, the video data is preprocessed and input into a generative model. The generative model detects specific behavioral patterns (e.g., putting an item into a bag) and outputs the results.

[0789] Step 3:

[0790] If the server detects abnormal behavior based on the output of the generative model, it generates an alert. This alert information is output and sent to the staff's terminal. The alert contains specific information such as "Suspicious behavior has been detected on shelf number 2."

[0791] Step 4:

[0792] The terminal displays the alert received from the server. The staff member checks the notification and prepares to quickly head to the relevant area. Specifically, the staff member checks the alert content on the screen and heads to the scene.

[0793] Step 5:

[0794] When abnormal behavior is detected, the server automatically generates a voice announcement using the generative model. The generated voice announcement is output. For example, it could say, "Attention customers, please be aware that security cameras are in operation."

[0795] Step 6:

[0796] The server collects listing information from major online marketplaces, which serves as input, stores the listing information in a database, and prepares it for analysis.

[0797] Step 7:

[0798] The server analyzes the collected listing information using a pattern recognition algorithm, detecting fraudulent listings based on the listing price and number of listings, and outputs the results.

[0799] Step 8:

[0800] If the server detects fraudulent listing activity, it generates a flag and sends a notification to the online platform administrator's device, stating that "a particular listing may be fraudulent."

[0801] Step 9:

[0802] The terminal displays the flag notification received from the server. The administrator checks the notification and sends a warning message to the problematic seller. Specifically, the administrator sends a message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0803] Step 10:

[0804] The user (administrator) can suspend any listings that are deemed fraudulent as necessary, and conduct additional detailed checks and investigations. Specifically, the user selects the relevant listing from the system's administration screen and clicks the suspend button.

[0805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0806] The security system of the present invention employs technology that combines a generative model and an emotion engine to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[0807] In-store monitoring and announcements

[0808] Video data collection and analysis

[0809] server

[0810] The server collects video data in real time from multiple cameras installed in the store. The video data received from the cameras is analyzed using image recognition algorithms and an emotion engine. For example, it detects specific behavioral patterns (e.g., hiding items in a bag, attempting to steal a large amount of merchandise) and changes in facial expressions (anxiety, impatience, etc.).

[0811] Alert generation and notification

[0812] server

[0813] When the server detects abnormal behavior or changes in emotion, it generates an alert based on that information, which includes the specific content and location of the detected behavior or emotion.

[0814] Terminal

[0815] The store staff's device receives the alert sent from the server and displays a notification, such as "Suspicious behavior and an anxious user expression were detected on shelf number 2."

[0816] Real-time announcements

[0817] server

[0818] The server uses the generative model and emotion engine to automatically generate voice announcements and broadcast them throughout the store. The content of the announcement changes dynamically depending on the user's emotional state. For example, a message such as "Customers, please be aware that security cameras are currently in operation." may be generated.

[0819] User (store staff)

[0820] Staff check the alert on their device and rush to the scene, where they can confirm the actual situation and take appropriate action. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[0821] Online resale monitoring

[0822] Collection and analysis of listing information

[0823] server

[0824] The server periodically collects listings from the online platform, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[0825] Fake listing detection and notification

[0826] server

[0827] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[0828] Terminal

[0829] The administrator's device receives the flag notification from the server and displays it, allowing the administrator to quickly identify any fraudulent listings.

[0830] Warning messages and pauses

[0831] User (Administrator)

[0832] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, a message such as "The price setting for this item is inappropriate. We will conduct a detailed investigation" will be sent. If necessary, the listing will be suspended and additional investigation will be carried out.

[0833] Specific examples

[0834] Specific examples of in-store monitoring and announcements

[0835] 1. Server: Analyzes in-store camera footage and detects suspicious behavior and anxious expressions from users. For example, if a customer puts a large number of items into a bag at once and looks anxious.

[0836] 2. Server: Generates an alert and notifies the staff member's device that "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf 2."

[0837] 3. Terminal: Staff check the alert on the terminal and rush to the area to check the situation.

[0838] 4. Server: Using the generative model and emotion engine, an automatically generated voice announcement is played in the store, informing customers, "Customers, please be aware that security cameras are in operation."

[0839] Examples of online resale monitoring

[0840] 1. Server: Collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a specific brand have been listed repeatedly at abnormally low prices.

[0841] 2. Server: Flags the listing as fraudulent and notifies the administrator's device.

[0842] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0843] 4. User (Administrator): Pause the listing and conduct additional detailed checks and investigations.

[0844] In this way, by combining emotion engines, the present invention achieves even more accurate fraud detection and response.

[0845] The processing flow will be explained below.

[0846] In-store monitoring and announcement processing flow

[0847] Step 1:

[0848] server

[0849] The server collects video data in real time from multiple cameras installed in the store, and the video data is immediately sent to the image recognition algorithm and emotion engine.

[0850] Step 2:

[0851] server

[0852] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[0853] Step 3:

[0854] server

[0855] The server uses an emotion engine to analyze the user's facial expressions, detecting, for example, changes in facial expression that indicate feelings of anxiety or impatience.

[0856] Step 4:

[0857] server

[0858] If any abnormal behavior or emotional changes are detected, the server immediately generates an alert, which includes the specific content and location of the detected behavior or emotion.

[0859] Step 5:

[0860] server

[0861] The server then sends the generated alert to the store staff's device, which contains a notification such as, "Suspicious behavior and anxious user expression were detected on shelf number 2."

[0862] Step 6:

[0863] Terminal

[0864] Store staff's devices receive and display alerts from the server, allowing them to quickly grasp the location and circumstances of suspicious activity.

[0865] Step 7:

[0866] User (store staff)

[0867] Staff check the alerts received on their devices, rush to the scene, check the actual situation there, and take appropriate action if necessary.

[0868] Step 8:

[0869] server

[0870] The server uses the generative model and emotion engine to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[0871] Step 9:

[0872] server

[0873] The generated audio announcements are played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[0874] Online Resale Monitoring Process Flow

[0875] Step 1:

[0876] server

[0877] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[0878] Step 2:

[0879] server

[0880] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[0881] Step 3:

[0882] server

[0883] The server uses an emotion engine to analyze changes in emotions from sellers' profiles and comments, detecting, for example, when a seller appears unnaturally anxious.

[0884] Step 4:

[0885] server

[0886] If a listing is deemed fraudulent or emotionally abnormal, the server flags the listing, which includes details about the listing and the abnormality.

[0887] Step 5:

[0888] server

[0889] The server then sends the generated flag to the terminal of the online platform administrator, which contains a notification such as "A particular listing may be fraudulent."

[0890] Step 6:

[0891] Terminal

[0892] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[0893] Step 7:

[0894] User (Administrator)

[0895] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[0896] Step 8:

[0897] User (Administrator)

[0898] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[0899] In this way, by combining the emotion engine, the system of the present invention can achieve even more accurate fraud detection and response.

[0900] Example 2

[0901] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Conventional security systems monitor in-store shoplifting and online fraudulent resale separately, making it impossible to manage both in an integrated manner. Furthermore, there is a need for more accurate monitoring and notification that takes into account the emotional state of the user, rather than simply detecting abnormal behavior through video analysis. Furthermore, more advanced analysis and rapid notification are required to detect fraudulent listings and respond appropriately.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0904] In this invention, the server includes means for analyzing video data using a generative model and detecting abnormal behavior and changes in emotion, means for generating audio announcements using the generative model and emotion engine and notifying the facility, and means for sending an alert to a staff member's terminal when abnormal behavior or changes in emotion are detected. This makes it possible to accurately detect fraudulent behavior within the facility and take appropriate action taking into account changes in emotion.

[0905] Additionally, for monitoring online resale, the system includes a means for collecting listing information and using pattern recognition algorithms to detect fraudulent listings, a means for sending warning messages to sellers determined to be fraudulent, and a means for suspending listings determined to be fraudulent, making it possible to quickly detect fraudulent online resale activities and take appropriate measures.

[0906] A "generative model" is an algorithm that uses machine learning and deep learning techniques to generate data such as text, images, and audio.

[0907] "Video Data" means real-time or recorded visual data collected by a camera or other image capture device.

[0908] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior, and includes behavior that is deemed suspicious for security purposes.

[0909] "Changes in emotions" refers to fluctuations in the psychological state that can be inferred from the facial expressions and attitudes of the person being observed.

[0910] An "emotion engine" is an algorithm that analyzes emotions from image and video data and identifies their state.

[0911] "Voice announcement" is an output means for generating a specific message as voice and announcing it through a speaker.

[0912] An "alert" is a notification issued when the system detects an abnormal condition or suspicious behavior, prompting a warning or action.

[0913] A "terminal" is an electronic device (such as a computer, tablet, or smartphone) that allows staff or managers to receive information and perform operations.

[0914] "Listing information" refers to data containing detailed information about products listed on online marketplaces and auction sites.

[0915] A "pattern recognition algorithm" is an algorithm that extracts regularities and patterns from data and analyzes anomalies and characteristics.

[0916] "Fraudulent listing" refers to the listing of a product that deviates from normal market prices and sales methods and violates standards and regulations.

[0917] A "warning message" is a notification sent to notify of fraudulent activity or an abnormal state, and includes content urging improvement.

[0918] "Suspension" refers to the temporary interruption or cessation of a system or operation.

[0919] The present invention is a security system for monitoring and preventing in-store and online fraud. The system uses generative models and emotion engines to detect and respond to in-store shoplifting and online fraudulent resale activities.

[0920] In-store monitoring and announcements

[0921] Server: Multiple cameras are installed in the store to collect video data in real time. Specifically, IP cameras are used to send video data to the server via RTSP (Real Time Streaming Protocol).

[0922] Server: The collected video data is analyzed in real time using image recognition algorithms (such as OpenCV or Amazon Rekognition) running on the cloud. An emotion engine such as Microsoft Azure's Emotion API is also used to analyze the user's facial expressions and detect abnormal behavior and changes in emotion. For example, the system can detect when a customer picks up an item from a shelf and puts it in a bag, or when an anxious expression appears.

[0923] Server: Generates alerts based on the detection results and sends them to store staff terminals. The alerts include the content and location of the detected behavior and changes in emotions.

[0924] Server: Creates the content of the voice announcement using a generative AI model (e.g., GPT-3). Dynamically changes the content of the announcement based on information from the emotion engine. For example, a message such as "Customers, please be aware that security cameras are in operation" is generated and announced through the in-store speaker system.

[0925] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and take action as necessary. For example, they ask the customer, "Excuse me, is there anything I can help you with?"

[0926] Online resale monitoring

[0927] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[0928] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when a high-priced product of a particular brand is repeatedly listed at an abnormally low price.

[0929] Server: Based on the analysis results, flags suspicious listings as fraudulent and notifies the administrator's device. The notification includes information about the product, price, seller, etc.

[0930] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the listing will be suspended and further investigation will be conducted.

[0931] User (Administrator): After sending a warning message, we will conduct additional detailed checks and investigations, such as checking past transaction history and other listings by the seller to verify whether there has been any ongoing fraudulent activity.

[0932] Examples of specific examples and prompts

[0933] Specific examples of in-store monitoring and announcements

[0934] 1. Server: Collects camera footage from within the store and detects suspicious behavior and anxious expressions of customers. Example: A customer picks up multiple items from a shelf and puts them in a bag.

[0935] 2. Server: Based on the detected results, an alert is generated stating, "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf number 2," and notified to the staff member's device.

[0936] 3. Terminal: Staff checks the alert on the terminal and rushes to the area to check the situation. Example: After staff arrives at the scene, they ask the customer, "Excuse me, is there anything I can help you with?"

[0937] 4. Server: Using the generative model and emotion engine, an audio announcement is played throughout the store saying, "Customers, please be aware that security cameras are in operation."

[0938] Examples of online resale monitoring

[0939] 1. Server: Collects listing information via the API of an e-commerce site and detects that expensive watches of a particular brand have been listed repeatedly at abnormally low prices.

[0940] 2. Server: Flags the listing as fraudulent and sends a notification to the administrator's device saying, "A specific brand of watch has been repeatedly listed at an abnormal price."

[0941] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[0942] 4. User (Administrator): Pause the item in question and investigate past sales history and other listings to identify any abnormal patterns.

[0943] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0944] In-store monitoring and announcements

[0945] Step 1: Collect video data

[0946] Server: Collects video data in real time from multiple cameras installed in the store. Specifically, the IP cameras send video data to the server using RTSP (Real Time Streaming Protocol).

[0947] Input: Real-time video data from IP cameras

[0948] Output: Raw video data stored on a server

[0949] Step 2: Analyzing the video data

[0950] Server: Analyzes collected video data using an image recognition algorithm (e.g., OpenCV or Amazon Rekognition) running on the cloud. It also analyzes the user's facial expressions using an emotion engine (e.g., Microsoft Azure's Emotion API). Detects when a customer exhibits certain suspicious behavior (e.g., hiding a product in a bag) or changes in emotion (e.g., anxiety, impatience, etc.).

[0951] Input: Raw video data

[0952] Data processing / data calculation: Analyzes video data using image recognition algorithms and emotion engines to detect suspicious behavior and changes in emotions

[0953] Output: Analysis results (presence or absence of suspicious behavior and emotional changes, location information)

[0954] Step 3: Generate alerts and notifications

[0955] Server: Based on the analysis results, if suspicious behavior or changes in emotion are detected, an alert is generated. The alert includes the details of the detected behavior, location, and changes in emotion. The generated alert is sent to the store staff's terminal.

[0956] Input: Analysis results

[0957] Data processing / data calculation: Alert information generation

[0958] Output: Alert notification sent to store staff's terminal

[0959] Step 4: Real-time announcements

[0960] Server: Uses a generative model (e.g., GPT-3) to create the content of the voice announcement. Dynamically changes the content of the announcement based on information from the emotion engine. For example, it generates a message such as "Customers, please be careful as security cameras are currently in operation," and announces it through the store's speaker system.

[0961] Input: Alert information, emotion engine analysis results

[0962] Data processing / data calculation: voice announcement generation

[0963] Output: Voice announcement played through the in-store speaker system

[0964] Step 5: On-site inspection

[0965] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and, if necessary, take appropriate action against the customer. For example, they might ask, "Excuse me, is there anything I can help you with?"

[0966] Input: Alert notification

[0967] Data processing / data calculation: On-site confirmation and response

[0968] Output: Response to suspicious behavior

[0969] Online resale monitoring

[0970] Step 1: Gather your listing information

[0971] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[0972] Input: Online Marketplace Listings API

[0973] Output: Listing data saved on the server

[0974] Step 2: Listing Analysis

[0975] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when expensive products of a particular brand are repeatedly listed at abnormally low prices.

[0976] Input: Listing information data

[0977] Data processing / data calculation: Analyzes data using pattern recognition algorithms to identify fraudulent listings

[0978] Output: Analysis results (whether or not there is fraudulent listing, seller information)

[0979] Step 3: Detect and notify fraudulent listings

[0980] Server: Based on the analysis results, the listing is flagged as fraudulent and the information is sent to the administrator's device. The notification includes information about the product, price, seller, etc.

[0981] Input: Analysis results

[0982] Data processing / data calculation: Alert information generation

[0983] Output: Alert notification sent to the administrator's device

[0984] Step 4: Send warning messages and suspend listings

[0985] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the administrator can also suspend the relevant listing.

[0986] Input: Alert notification

[0987] Data processing / data calculation: generating warning messages and suspending listings

[0988] Output: Warning message sent to seller, listing suspension status

[0989] Step 5: Further investigation

[0990] User (Administrator): After sending a warning message to the seller, conduct additional detailed checks and investigations, such as checking past transaction history and other listings to verify whether there is ongoing fraudulent activity.

[0991] Input: Seller's transaction history, past listing information

[0992] Data processing / data calculation: Analysis of transaction history and listing information

[0993] Output: Report of findings and required actions

[0994] (Application example 2)

[0995] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0996] Preventing shoplifting and vandalism is crucial for modern stores, but many current security systems have difficulty quickly and accurately detecting suspicious behavior or changes in user emotion. Furthermore, fraudulent listings on online platforms are on the rise, requiring significant effort to monitor and respond to. To solve these problems, a security system that combines more advanced technologies is needed.

[0997] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0998] In this invention, the server includes a means for analyzing video data using a generative model to detect abnormal behavior and changes in facial expression, a means for generating audio announcements using the generative model and notifying the store, a means for notifying staff of an alert when abnormal behavior or changes in facial expression are detected, a means for conducting on-site inspections based on the alert notified to the staff, and a means for analyzing the user's emotional state using an emotion engine and generating an appropriate warning message. This makes it possible to quickly and accurately detect suspicious behavior and changes in facial expression and take appropriate action. It also makes it possible to detect fraudulent listings online and quickly respond to them.

[0999] A "generative model" is a type of machine learning algorithm that learns patterns from collected data and makes inferences based on new data.

[1000] "Video data" is a data format that refers to images and video information captured by cameras and other visual sensors.

[1001] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and indicates misconduct or risky behavior.

[1002] "Changes in facial expression" refers to changes in emotional state indicated by facial muscle movements, and is used to detect abnormal states such as anxiety or impatience.

[1003] "Voice announcement" is a means of conveying information through voice, and uses a generative model to automatically generate messages appropriate to the situation.

[1004] An "alert" refers to a warning message or notification issued when abnormal behavior or conditions are detected.

[1005] "Staff" refers to employees who are engaged in the operation of stores and facilities and who are responsible for safety management and customer service.

[1006] An "emotion engine" refers to technology that analyzes a user's facial expressions and behavioral data to determine their emotional state.

[1007] A "warning message" is a written or audio message that alerts a target or administrator when fraud or an abnormal condition is detected.

[1008] "Online platform" refers to a system or website for conducting transactions or exchanging information over the Internet.

[1009] "Fraudulent listing" refers to the act of selling products on online platforms with inappropriate pricing or false information.

[1010] A "pattern recognition algorithm" is a technology that automatically identifies specific patterns in data and is used to analyze image and text data.

[1011] "Listing information" refers to data on an online platform that lists product details, prices, etc.

[1012] The security system of the present invention uses a generative model and an emotion engine to monitor in-store security and fraudulent listings on online platforms. A specific system configuration and processing method for implementing the present invention will be described in detail below.

[1013] System Configuration

[1014] Hardware Configuration

[1015] 1. Server:

[1016] A high-performance data analysis server (e.g., an Amazon EC2 instance).

[1017] 2. Camera:

[1018] High-resolution surveillance cameras (e.g. network cameras).

[1019] 3. Terminal:

[1020] An iOS or Android smartphone, or AR-enabled glasses (e.g., Google Glass).

[1021] Software Configuration

[1022] 1. Image Recognition Algorithm:

[1023] Uses OpenCV and TensorFlow.

[1024] 2. Emotion Engine:

[1025] It uses AWS's Amazon Rekognition and Microsoft Azure's Emotion API.

[1026] 3. Generative Model:

[1027] OpenAI's GPT-3.

[1028] Processing steps

[1029] In-store surveillance

[1030] 1. Camera footage collection and analysis:

[1031] Multiple cameras installed in the store transmit video in real time to a server, which then analyzes the video data using OpenCV and TensorFlow to detect specific behavioral patterns and changes in facial expressions.

[1032] 2. Alert generation and notification:

[1033] The server analyzes the video data using an emotion engine (AWS Rekognition) to determine the user's emotional state. If suspicious behavior or abnormal emotions are detected, an alert is generated and sent to the staff member's smartphone or smart glasses.

[1034] 3. Generate voice announcements:

[1035] If abnormal behavior or emotional changes are detected, the server uses OpenAI's GPT-3 to generate appropriate voice announcements, such as "Customers, please be aware that security cameras are in operation," which are broadcast throughout the store.

[1036] Online resale monitoring

[1037] 1. Listing information collection and analysis:

[1038] The server periodically collects listing information from online platforms and uses a pattern recognition algorithm to detect when a particular brand of high-priced product is repeatedly listed at an abnormally low price.

[1039] 2. Fake listing detection and notification:

[1040] The server flags the listing as fraudulent and notifies the administrator's terminal, who then checks the notification and sends a warning message to the seller.

[1041] Specific examples

[1042] 1. Camera footage analysis:

[1043] "The server receives video data from cameras in the store in real time. The received video data is analyzed using image recognition algorithms using OpenCV and TensorFlow. For example, it detects specific behavioral patterns (e.g., hiding products in a bag)."

[1044] 2. Sentiment analysis and alert generation:

[1045] "The received video data is analyzed using AWS's Amazon Rekognition to analyze the user's emotional state. If abnormal emotions such as anxiety or impatience are detected, the server generates an alert based on that information and sends it to the staff member's smartphone or smart glasses."

[1046] 3. Generate voice announcements:

[1047] "After detecting suspicious behavior or emotions, the server automatically generates an appropriate voice announcement using OpenAI's GPT-3. For example, a message such as, 'Customers, please be aware that security cameras are in operation, so please be careful,' is generated and broadcast throughout the store."

[1048] 4. Examples of prompts:

[1049] "Detect suspicious activity from the following camera footage and generate prompts to create alert messages corresponding to the activity:

[1050] Camera footage shows the customer carelessly putting a large amount of items into a bag and frequently looking back. What warning message corresponds to this behavior?

[1051] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1052] Step 1:

[1053] The server collects video data in real time from cameras installed in the store. The video data is transferred from the cameras to the server via a network. The video data is the input, and it becomes the basis for subsequent analysis processing.

[1054] Step 2:

[1055] The server analyzes the collected video data using OpenCV and TensorFlow. Specifically, it recognizes objects in the video and detects behavioral patterns. This allows it to identify abnormal behaviors and specific behavioral patterns. The output is data that includes abnormal behaviors and specific behaviors.

[1056] Step 3:

[1057] The server performs emotion analysis on the video data using AWS's Amazon Rekognition. It analyzes the facial expressions of people in the video and detects emotions such as anxiety or impatience. The input is the video data obtained in Step 2, and the output is the result of the emotion analysis.

[1058] Step 4:

[1059] If abnormal behavior or abnormal emotion is detected, the server generates an alert. The alert includes the content and location of the abnormal behavior or emotion. The alert generation includes a means to generate appropriate messages and warning content based on the analysis results. The output is alert notification data to staff.

[1060] Step 5:

[1061] The server sends the generated alert to the staff member's device (smartphone or smart glasses) via the network. The staff member's device receives the notification and displays it on its screen. The input is the generated alert data, and the output is the notification displayed on the staff member's device.

[1062] Step 6:

[1063] The server generates voice announcements using OpenAI's GPT-3. Based on the analysis of abnormal behavior and emotions, an appropriate warning message is automatically generated. The output is the generated voice announcement data.

[1064] Step 7:

[1065] The voice announcement data is broadcast through the sound system in the store. This allows everyone in the store to be alerted. The input is the voice announcement data generated in Step 6, and the output is the voice announcement from the sound system in the store.

[1066] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1067] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1068] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1069] [Third embodiment]

[1070] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1071] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1072] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1073] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1074] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1075] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1076] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1077] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1078] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1079] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1080] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1081] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1082] The security system of the present invention employs technology that uses generative models to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[1083] In-store monitoring and announcements

[1084] Video data collection and analysis

[1085] server

[1086] The server collects video data in real time from multiple cameras in the store. The video data received from the cameras is analyzed using image recognition algorithms. For example, trained models are used to detect specific behavioral patterns (e.g., putting items into a bag or attempting to take away a large number of items at once).

[1087] Alert generation and notification

[1088] server

[1089] When the server detects abnormal behavior, it generates an alert based on that information, which is then sent to the store staff's terminal.

[1090] Terminal

[1091] The store staff's devices receive the alert sent from the server and display a notification, such as "Suspicious activity has been detected on shelf number 2."

[1092] Real-time announcements

[1093] server

[1094] The server uses the generative model to automatically generate voice announcements and broadcast them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation."

[1095] User (store staff)

[1096] Staff can check the alert on the terminal and quickly move to the relevant area to check the situation. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[1097] Online resale monitoring

[1098] Collection and analysis of listing information

[1099] server

[1100] The server automatically collects listings from online platforms, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[1101] Fake listing detection and notification

[1102] server

[1103] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[1104] Terminal

[1105] The administrator's device receives the flag notification from the server and displays the notification, such as "A particular listing may be fraudulent."

[1106] Warning messages and pauses

[1107] User (Administrator)

[1108] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, they can send a message saying, "This listing is being temporarily suspended due to the possibility of fraudulent transactions." If necessary, they can temporarily suspend the listing and conduct additional investigations.

[1109] Specific examples

[1110] Specific examples of in-store monitoring and announcements

[1111] 1. The server analyzes camera footage in the store and detects suspicious behavior, such as a customer putting a large number of items into a bag at once.

[1112] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[1113] 3. Staff check the alert on their device, go to the relevant area and check the situation on site.

[1114] 4. At the same time, the server automatically generates an audio announcement and plays it throughout the store to raise awareness of crime prevention.

[1115] Examples of online resale monitoring

[1116] 1. The server collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a particular brand have been listed repeatedly at abnormally low prices.

[1117] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[1118] 3. The administrator checks the notification on the device and sends a warning message to the seller stating, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1119] 4. The administrator will suspend the relevant listing and conduct further detailed checks and investigations.

[1120] The above is an example of a system of the present invention that can effectively prevent fraud both in-store and online.

[1121] The processing flow will be explained below.

[1122] In-store monitoring and announcement processing flow

[1123] Step 1:

[1124] server

[1125] The server collects video data in real time from cameras installed in the store, and the video data is immediately sent to an image recognition algorithm.

[1126] Step 2:

[1127] server

[1128] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[1129] Step 3:

[1130] server

[1131] If any abnormal behavior is detected, the server immediately generates an alert, which includes the specific details and location of the detected behavior.

[1132] Step 4:

[1133] server

[1134] The server then sends the generated alert to the store staff's device, which contains a notification such as "Suspicious activity has been detected on shelf number 2."

[1135] Step 5:

[1136] Terminal

[1137] Store staff's devices receive and display alerts from the server, allowing them to quickly identify the location of any suspicious activity.

[1138] Step 6:

[1139] User (store staff)

[1140] Staff check the alerts received on their devices, rush to the scene, confirm the actual situation, and take appropriate action.

[1141] Step 7:

[1142] server

[1143] The server uses the generative model to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[1144] Step 8:

[1145] server

[1146] The generated audio announcement is played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[1147] Online Resale Monitoring Process Flow

[1148] Step 1:

[1149] server

[1150] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[1151] Step 2:

[1152] server

[1153] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[1154] Step 3:

[1155] server

[1156] If a listing is determined to be fraudulent, the server will flag the listing, which will include details about the listing and any anomalies.

[1157] Step 4:

[1158] server

[1159] The server then sends the generated flag to the online platform administrator's terminal, which contains a notification such as "A particular listing may be fraudulent."

[1160] Step 5:

[1161] Terminal

[1162] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[1163] Step 6:

[1164] User (Administrator)

[1165] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1166] Step 7:

[1167] User (Administrator)

[1168] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[1169] In this way, the system of the present invention can efficiently implement security measures both offline and online.

[1170] Example 1

[1171] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1172] Preventing fraudulent activities is a major challenge in modern commercial facilities. Detecting and responding to shoplifting and fraudulent resale activities in real time is particularly challenging. Addressing these challenges requires building an efficient and accurate surveillance system. However, conventional systems have struggled to quickly and effectively detect and respond to these fraudulent activities. Therefore, the present invention aims to provide a new system that utilizes generative models to detect fraudulent activities in real time and respond quickly to them.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1174] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the facility, and means for notifying workers of an alert when abnormal behavior is detected. This makes it possible to detect abnormal behavior such as shoplifting in real time and prevent fraudulent activity through audio announcements. Furthermore, the server includes means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, means for sending warning messages to sellers determined to be fraudulent, and means for suspending listings determined to be fraudulent, thereby making it possible to effectively monitor and prevent fraudulent online resale activities.

[1175] A "generative model" is a model that uses machine learning algorithms to generate information or patterns from data.

[1176] "Video data" refers to data containing visual information obtained from a camera, video recording device, or the like.

[1177] "Abnormal behavior" is behavior that deviates from normal patterns of behavior and indicates misconduct or behavior that requires attention.

[1178] A "voice announcement" is a voice message automatically generated using voice synthesis technology.

[1179] An "alert" is a warning notification sent when the system detects an abnormality.

[1180] "Worker" refers to a person in charge of monitoring and responding within a store or facility.

[1181] "Listing Information" means the information provided when a product is offered for sale on an online platform.

[1182] A "pattern recognition algorithm" is an algorithm that identifies specific patterns or features in data.

[1183] A "warning message" is a message sent to notify of irregularities or abnormalities.

[1184] "Pause" is the act of temporarily halting a particular action or process.

[1185] The system of the present invention employs techniques that leverage generative models and pattern recognition algorithms to effectively monitor and prevent fraud in-store and online.

[1186] In-store monitoring and announcements

[1187] Video data collection and analysis

[1188] The server collects video data in real time from cameras installed in the store. The cameras are installed on the ceiling and shelves, and the viewing angle and resolution are set according to the store's layout. The collected video data is analyzed using image recognition algorithms such as the YOLO model using TensorFlow. Specifically, suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once) are detected.

[1189] Alert generation and notification

[1190] When the server detects suspicious behavior, it generates an alert and sends the information in JSON format to the store staff's device. The transmission is carried out using a REST API using the HTTP protocol. The device displays the received alert as a pop-up notification, for example, saying, "Suspicious behavior has been detected on shelf number 2."

[1191] Real-time announcements

[1192] The server automatically generates a voice announcement using a generative model (e.g., Google Text-to-Speech API) and broadcasts it throughout the store. The announcement includes a warning message such as, "Security cameras are in operation, please be careful." The user (store staff) checks the alert and quickly moves to the relevant area to check the situation.

[1193] Online resale monitoring

[1194] Collection and analysis of listing information

[1195] The server periodically collects listing information from major online platforms using scraping technology. The collected information is then analyzed using pattern recognition algorithms, such as logistic regression models using scikit-learn, to identify fraudulent listings. For example, pricing that deviates significantly from normal market prices or a large number of listings in a short period of time are monitored.

[1196] Fake listing detection and notification

[1197] The server generates a "fraudulent" flag for listings that are determined to be fraudulent and sends that information in JSON format to the online platform administrator's device. The notification includes a message such as "A specific listing may be fraudulent." The device then displays the received flag as a pop-up notification.

[1198] Sending warning messages and suspending listings

[1199] The user (administrator) checks the notification and sends a warning message to the relevant seller, saying, "This listing is temporarily suspended due to the possibility of fraudulent transactions." The user also temporarily suspends the listing through the system interface and conducts additional investigation.

[1200] Specific examples

[1201] Specific examples of in-store monitoring and announcements

[1202] 1. The server analyzes camera footage in the store and detects customers putting large amounts of items into bags at once.

[1203] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[1204] 3. The user (store staff member) checks the alert on the device, goes to the relevant area and checks the situation on site.

[1205] 4. The server automatically generates voice announcements and plays them throughout the store to raise awareness of crime prevention.

[1206] Examples of online resale monitoring

[1207] 1. The server collects listing information from e-commerce sites and uses a pattern recognition algorithm to detect when expensive watches from a particular brand are repeatedly listed at abnormally low prices.

[1208] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[1209] 3. The user (administrator) checks the notification on their device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1210] 4. The user (administrator) will suspend the relevant listing and conduct additional detailed checks and investigations.

[1211] The above is an embodiment of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in-store and online.

[1212] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1213] In-store monitoring and announcements

[1214] Step 1: Collect video data

[1215] The server collects video data in real time from cameras installed in the store, such as those on the ceiling or on shelves, with the viewing angle and resolution set according to the store's layout.

[1216] Input: Video data from the camera

[1217] Output: Raw video data sent to the server

[1218] Step 2: Analyzing the video data

[1219] The server analyzes the collected video data using image recognition algorithms such as the YOLO model powered by TensorFlow, specifically detecting suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once).

[1220] Input: Raw video data stored on the server

[1221] Output: Analyzed video data and suspicious behavior detection results

[1222] Step 3: Generate an alert

[1223] If the server detects any suspicious activity as a result of the analysis, it generates an alert, which includes the camera's location and timestamp.

[1224] Input: Suspicious behavior detection result

[1225] Output: Alert information (location, timestamp, etc.)

[1226] Step 4: Alert Notification

[1227] The server sends the generated alert to the store staff's terminal using the REST API with the HTTP protocol. The transmission format is JSON. The terminal displays the received alert as a pop-up notification.

[1228] Input: Alert information

[1229] Output: Notification displayed on device (e.g. "Suspicious activity detected on shelf 2")

[1230] Step 5: Generate real-time announcements

[1231] The server uses a generative model (e.g., Google Text-to-Speech API) to generate audio announcements, such as "Security cameras are active, please be careful."

[1232] Input: Alert information

[1233] Output: Generated voice announcement data

[1234] Step 6: Sending real-time announcements

[1235] The server transmits the generated voice data to a speaker system in the store, and the announcement is broadcast throughout the store.

[1236] Input: Generated voice announcement data

[1237] Output: Announcement played in store

[1238] Step 7: Staff Response

[1239] The user (store staff) checks the alert on the terminal, quickly moves to the relevant area, and checks the situation on the spot. If necessary, they can also play back and check the recorded data from the surveillance camera.

[1240] Input: Alert notification displayed on the terminal

[1241] Output: On-site confirmation and response

[1242] Online resale monitoring

[1243] Step 1: Gather your listing information

[1244] The server periodically collects listing information from major online platforms using scraping technology, such as the Python library Scrapy.

[1245] Input: Online platform webpage information

[1246] Output: Listing information data (product name, price, category, seller information, etc.)

[1247] Step 2: Listing Analysis

[1248] The server analyzes the collected listing information using pattern recognition algorithms such as logistic regression models using scikit-learn to identify fraudulent listings, such as those with prices that deviate significantly from normal market prices or those with a large number of listings in a short period of time.

[1249] Input: Listing information data

[1250] Output: Identification results of fraudulent listings (listing ID, price, reason for detection, etc.)

[1251] Step 3: Generate flags

[1252] Based on the analysis results, the server generates a "Fraud" flag for fraudulent listings. This flag contains information such as the seller ID, listing ID, and the reason for detection.

[1253] Input: Result of identifying fraudulent listings

[1254] Output: Invalid flag information

[1255] Step 4: Flag Notification

[1256] The server notifies the online platform administrator of the generated fraud flag via a REST API using the HTTP protocol. The transmission format is JSON. The terminal displays the received flag as a pop-up notification.

[1257] Input: Invalid flag information

[1258] Output: Notification displayed on device (e.g. "This particular listing may be fraudulent")

[1259] Step 5: Sending a warning message

[1260] The user (administrator) checks the notification and sends a warning message to the relevant seller, such as "This listing is temporarily suspended as it may be a fraudulent transaction."

[1261] Input: Invalid flag information

[1262] Output: Warning message sent to seller

[1263] Step 6: Pause your listing

[1264] The user (administrator) temporarily suspends the relevant listing through the system interface, and takes action to stop the listing from being published in order to conduct additional investigation.

[1265] Input: Invalid flag information

[1266] Output: Paused listings

[1267] Step 7: Conduct additional research

[1268] The user (administrator) will check the details of the suspected fraudulent listing and conduct further investigation if necessary, for example, by checking the seller's past transaction data or other listed items.

[1269] Enter: Paused Listings

[1270] Output: Investigation results (determination of the suitability of the listing)

[1271] The above are the specific processing steps of the system of the present invention, which enable effective monitoring and prevention of fraudulent activities in stores and online.

[1272] (Application example 1)

[1273] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1274] Conventional security systems did not provide sufficient means to prevent shoplifting and unauthorized resales occurring in stores. This resulted in a heavy burden on staff and reduced security efficiency. Furthermore, manual monitoring and response was required for online unauthorized resales, making it difficult to detect fraudulent activity early. An efficient and effective solution to these issues was needed.

[1275] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1276] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the store, means for notifying staff of an alert when abnormal behavior is detected, means for conducting on-site inspections based on the alert notified to the staff, and means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, thereby enabling effective monitoring and prevention of fraudulent activities both in-store and online.

[1277] A "generative model" is a model that uses machine learning algorithms to automatically learn patterns from data and perform specific tasks.

[1278] "Video data" refers to image and video information collected using a camera.

[1279] "Abnormal behavior" refers to actions or behavior that deviate from pre-established normal behavior patterns.

[1280] A "pattern recognition algorithm" is an algorithm for identifying and classifying specific patterns in data.

[1281] "Voice announcements" refer to audio notifications and guidance that are automatically generated using generative models.

[1282] An "alert" refers to a warning or notification sent to staff when the system detects abnormal behavior.

[1283] "Staff" refers to employees working in the store and those in charge of monitoring.

[1284] "Listing information" refers to detailed information about products sold on online marketplaces and e-commerce sites.

[1285] "Fraudulent listings" refer to products that are sold at prices significantly different from market value or through abnormal sales patterns.

[1286] "Warning Message" means a message sent to a seller or staff member to alert them to possible fraudulent activity.

[1287] The present invention is a security system that aims to efficiently detect and prevent fraudulent activities in stores and online using a generative AI model. Specific embodiments are described below.

[1288] In-store surveillance system

[1289] The server collects video data in real time from multiple cameras in the store. This video data is analyzed using image recognition algorithms (e.g., OpenCV or Caffe models). A trained generative model is used to detect specific behavioral patterns (e.g., putting items into a bag or trying to take away a large number of items at once).

[1290] When the server detects abnormal behavior, it immediately generates an alert and sends a notification to the store staff's device (such as a smartphone or tablet). The staff's device displays specific information, such as "Suspicious behavior has been detected on shelf number 2." The staff member checks the notification and quickly moves to the area in question to check the situation.

[1291] Furthermore, the server automatically generates voice announcements and sends them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation." This voice announcement will raise security awareness and is expected to have a deterrent effect on crime.

[1292] Online Resale Monitoring System

[1293] The server automatically collects listing information from major online marketplaces, which are then analyzed using pattern recognition algorithms to analyze information such as listing pricing and number of listings to identify listings that deviate from normal market prices and sales practices.

[1294] If the server detects fraudulent listing activity based on the analysis results, it generates a flag for the activity and sends it to the online platform administrator's device. The administrator's device displays a notification such as, "A particular listing may be fraudulent." The administrator then checks the notification and sends a warning message to the problematic seller. For example, the message might say, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1295] If necessary, the administrators will suspend the listing and conduct further detailed checks and investigations, which will enable them to quickly prevent unauthorized resale activities on the online platform.

[1296] Hardware and software used

[1297] Hardware used: security cameras, servers, smartphones, tablets

[1298] Software used: OpenCV (image recognition library), Caffe (deep learning model), REST API, push notification, Requests (HTTP request library)

[1299] Specific examples

[1300] For example, if there is ongoing fraudulent purchases or unfairly low sales of a particular product, the system will send a warning message such as, "The pricing of this product is inappropriate. We will conduct a detailed investigation."

[1301] Prompt Sentence Examples

[1302] "It uses image recognition algorithms to detect abnormal behavior in security camera footage in real time."

[1303] "We collect listing information from online platforms, detect abnormal behavioral patterns and generate alerts."

[1304] The above is a specific embodiment for carrying out the present invention, which enhances security both in-store and online, and enables rapid detection and response to fraudulent activity.

[1305] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1306] Step 1:

[1307] The server collects video data in real time from multiple cameras in the store. The video data sent from the cameras is input. The server stores the video data and prepares it for analysis.

[1308] Step 2:

[1309] The server analyzes the collected video data using image recognition algorithms (OpenCV and Caffe models). Specifically, the video data is preprocessed and input into a generative model. The generative model detects specific behavioral patterns (e.g., putting an item into a bag) and outputs the results.

[1310] Step 3:

[1311] If the server detects abnormal behavior based on the output of the generative model, it generates an alert. This alert information is output and sent to the staff's terminal. The alert contains specific information such as "Suspicious behavior has been detected on shelf number 2."

[1312] Step 4:

[1313] The terminal displays the alert received from the server. The staff member checks the notification and prepares to quickly head to the relevant area. Specifically, the staff member checks the alert content on the screen and heads to the scene.

[1314] Step 5:

[1315] When abnormal behavior is detected, the server automatically generates a voice announcement using the generative model. The generated voice announcement is output. For example, it could say, "Attention customers, please be aware that security cameras are in operation."

[1316] Step 6:

[1317] The server collects listing information from major online marketplaces, which serves as input, stores the listing information in a database, and prepares it for analysis.

[1318] Step 7:

[1319] The server analyzes the collected listing information using a pattern recognition algorithm, detecting fraudulent listings based on the listing price and number of listings, and outputs the results.

[1320] Step 8:

[1321] If the server detects fraudulent listing activity, it generates a flag and sends a notification to the online platform administrator's device, stating that "a particular listing may be fraudulent."

[1322] Step 9:

[1323] The terminal displays the flag notification received from the server. The administrator checks the notification and sends a warning message to the problematic seller. Specifically, the administrator sends a message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1324] Step 10:

[1325] The user (administrator) can suspend any listings that are deemed fraudulent as necessary, and conduct additional detailed checks and investigations. Specifically, the user selects the relevant listing from the system's administration screen and clicks the suspend button.

[1326] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1327] The security system of the present invention employs technology that combines a generative model and an emotion engine to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[1328] In-store monitoring and announcements

[1329] Video data collection and analysis

[1330] server

[1331] The server collects video data in real time from multiple cameras installed in the store. The video data received from the cameras is analyzed using image recognition algorithms and an emotion engine. For example, it detects specific behavioral patterns (e.g., hiding items in a bag, attempting to steal a large amount of merchandise) and changes in facial expressions (anxiety, impatience, etc.).

[1332] Alert generation and notification

[1333] server

[1334] When the server detects abnormal behavior or changes in emotion, it generates an alert based on that information, which includes the specific content and location of the detected behavior or emotion.

[1335] Terminal

[1336] The store staff's device receives the alert sent from the server and displays a notification, such as "Suspicious behavior and an anxious user expression were detected on shelf number 2."

[1337] Real-time announcements

[1338] server

[1339] The server uses the generative model and emotion engine to automatically generate voice announcements and broadcast them throughout the store. The content of the announcement changes dynamically depending on the user's emotional state. For example, a message such as "Customers, please be aware that security cameras are currently in operation." may be generated.

[1340] User (store staff)

[1341] Staff check the alert on their device and rush to the scene, where they can confirm the actual situation and take appropriate action. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[1342] Online resale monitoring

[1343] Collection and analysis of listing information

[1344] server

[1345] The server periodically collects listings from the online platform, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[1346] Fake listing detection and notification

[1347] server

[1348] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[1349] Terminal

[1350] The administrator's device receives the flag notification from the server and displays it, allowing the administrator to quickly identify any fraudulent listings.

[1351] Warning messages and pauses

[1352] User (Administrator)

[1353] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, a message such as "The price setting for this item is inappropriate. We will conduct a detailed investigation" will be sent. If necessary, the listing will be suspended and additional investigation will be carried out.

[1354] Specific examples

[1355] Specific examples of in-store monitoring and announcements

[1356] 1. Server: Analyzes in-store camera footage and detects suspicious behavior and anxious expressions from users. For example, if a customer puts a large number of items into a bag at once and looks anxious.

[1357] 2. Server: Generates an alert and notifies the staff member's device that "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf 2."

[1358] 3. Terminal: Staff check the alert on the terminal and rush to the area to check the situation.

[1359] 4. Server: Using the generative model and emotion engine, an automatically generated voice announcement is played in the store, informing customers, "Customers, please be aware that security cameras are in operation."

[1360] Examples of online resale monitoring

[1361] 1. Server: Collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a specific brand have been listed repeatedly at abnormally low prices.

[1362] 2. Server: Flags the listing as fraudulent and notifies the administrator's device.

[1363] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1364] 4. User (Administrator): Pause the listing and conduct additional detailed checks and investigations.

[1365] In this way, by combining emotion engines, the present invention achieves even more accurate fraud detection and response.

[1366] The processing flow will be explained below.

[1367] In-store monitoring and announcement processing flow

[1368] Step 1:

[1369] server

[1370] The server collects video data in real time from multiple cameras installed in the store, and the video data is immediately sent to the image recognition algorithm and emotion engine.

[1371] Step 2:

[1372] server

[1373] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[1374] Step 3:

[1375] server

[1376] The server uses an emotion engine to analyze the user's facial expressions, detecting, for example, changes in facial expression that indicate feelings of anxiety or impatience.

[1377] Step 4:

[1378] server

[1379] If any abnormal behavior or emotional changes are detected, the server immediately generates an alert, which includes the specific content and location of the detected behavior or emotion.

[1380] Step 5:

[1381] server

[1382] The server then sends the generated alert to the store staff's device, which contains a notification such as, "Suspicious behavior and anxious user expression were detected on shelf number 2."

[1383] Step 6:

[1384] Terminal

[1385] Store staff's devices receive and display alerts from the server, allowing them to quickly grasp the location and circumstances of suspicious activity.

[1386] Step 7:

[1387] User (store staff)

[1388] Staff check the alerts received on their devices, rush to the scene, check the actual situation there, and take appropriate action if necessary.

[1389] Step 8:

[1390] server

[1391] The server uses the generative model and emotion engine to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[1392] Step 9:

[1393] server

[1394] The generated audio announcement is played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[1395] Online Resale Monitoring Process Flow

[1396] Step 1:

[1397] server

[1398] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[1399] Step 2:

[1400] server

[1401] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[1402] Step 3:

[1403] server

[1404] The server uses an emotion engine to analyze changes in emotions from sellers' profiles and comments, detecting, for example, when a seller appears unnaturally anxious.

[1405] Step 4:

[1406] server

[1407] If a listing is deemed fraudulent or emotionally abnormal, the server flags the listing, which includes details about the listing and the abnormality.

[1408] Step 5:

[1409] server

[1410] The server then sends the generated flag to the online platform administrator's terminal, which contains a notification such as "A particular listing may be fraudulent."

[1411] Step 6:

[1412] Terminal

[1413] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[1414] Step 7:

[1415] User (Administrator)

[1416] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1417] Step 8:

[1418] User (Administrator)

[1419] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[1420] In this way, by combining the emotion engine, the system of the present invention can achieve even more accurate fraud detection and response.

[1421] Example 2

[1422] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1423] Conventional security systems monitor in-store shoplifting and online fraudulent resale separately, making it impossible to manage both in an integrated manner. Furthermore, there is a need for more accurate monitoring and notification that takes into account the emotional state of the user, rather than simply detecting abnormal behavior through video analysis. Furthermore, more advanced analysis and rapid notification are required to detect fraudulent listings and respond appropriately.

[1424] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1425] In this invention, the server includes means for analyzing video data using a generative model and detecting abnormal behavior and changes in emotion, means for generating audio announcements using the generative model and emotion engine and notifying the facility, and means for sending an alert to a staff member's terminal when abnormal behavior or changes in emotion are detected. This makes it possible to accurately detect fraudulent behavior within the facility and take appropriate action taking into account changes in emotion.

[1426] Additionally, for monitoring online resale, the system includes a means for collecting listing information and using pattern recognition algorithms to detect fraudulent listings, a means for sending warning messages to sellers determined to be fraudulent, and a means for suspending listings determined to be fraudulent, making it possible to quickly detect fraudulent online resale activities and take appropriate measures.

[1427] A "generative model" is an algorithm that uses machine learning and deep learning techniques to generate data such as text, images, and audio.

[1428] "Video Data" means real-time or recorded visual data collected by a camera or other image capture device.

[1429] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior, and includes behavior that is deemed suspicious for security purposes.

[1430] "Changes in emotions" refers to fluctuations in the psychological state that can be inferred from the facial expressions and attitudes of the person being observed.

[1431] An "emotion engine" is an algorithm that analyzes emotions from image and video data and identifies their state.

[1432] "Voice announcement" is an output means for generating a specific message as voice and announcing it through a speaker.

[1433] An "alert" is a notification issued when the system detects an abnormal condition or suspicious behavior, prompting a warning or action.

[1434] A "terminal" is an electronic device (such as a computer, tablet, or smartphone) that allows staff or managers to receive information and perform operations.

[1435] "Listing information" refers to data containing detailed information about products listed on online marketplaces and auction sites.

[1436] A "pattern recognition algorithm" is an algorithm that extracts regularities and patterns from data and analyzes anomalies and characteristics.

[1437] "Fraudulent listing" refers to the listing of a product that deviates from normal market prices and sales methods and violates standards and regulations.

[1438] A "warning message" is a notification sent to notify of fraudulent activity or an abnormal state, and includes content urging improvement.

[1439] "Suspension" refers to the temporary interruption or cessation of a system or operation.

[1440] The present invention is a security system for monitoring and preventing in-store and online fraud. The system uses generative models and emotion engines to detect and respond to in-store shoplifting and online fraudulent resale activities.

[1441] In-store monitoring and announcements

[1442] Server: Multiple cameras are installed in the store to collect video data in real time. Specifically, IP cameras are used to send video data to the server via RTSP (Real Time Streaming Protocol).

[1443] Server: The collected video data is analyzed in real time using image recognition algorithms (such as OpenCV or Amazon Rekognition) running on the cloud. An emotion engine such as Microsoft Azure's Emotion API is also used to analyze the user's facial expressions and detect abnormal behavior and changes in emotion. For example, the system can detect when a customer picks up an item from a shelf and puts it in a bag, or when an anxious expression appears.

[1444] Server: Generates alerts based on the detection results and sends them to store staff terminals. The alerts include the content and location of the detected behavior and changes in emotions.

[1445] Server: Creates the content of the voice announcement using a generative AI model (e.g., GPT-3). Dynamically changes the content of the announcement based on information from the emotion engine. For example, a message such as "Customers, please be aware that security cameras are in operation" is generated and announced through the in-store speaker system.

[1446] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and take action as necessary. For example, they ask the customer, "Excuse me, is there anything I can help you with?"

[1447] Online resale monitoring

[1448] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[1449] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when a high-priced product of a particular brand is repeatedly listed at an abnormally low price.

[1450] Server: Based on the analysis results, flags suspicious listings as fraudulent and notifies the administrator's device. The notification includes information about the product, price, seller, etc.

[1451] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the listing will be suspended and further investigation will be conducted.

[1452] User (Administrator): After sending a warning message, we will conduct additional detailed checks and investigations, such as checking past transaction history and other listings by the seller to verify whether there has been any ongoing fraudulent activity.

[1453] Examples of concrete examples and prompts

[1454] Specific examples of in-store monitoring and announcements

[1455] 1. Server: Collects camera footage from within the store and detects suspicious behavior and anxious expressions of customers. Example: A customer picks up multiple items from a shelf and puts them in a bag.

[1456] 2. Server: Based on the detected results, an alert is generated stating, "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf number 2," and notified to the staff member's device.

[1457] 3. Terminal: Staff checks the alert on the terminal and rushes to the area to check the situation. Example: After staff arrives at the scene, they ask the customer, "Excuse me, is there anything I can help you with?"

[1458] 4. Server: Using the generative model and emotion engine, an audio announcement is played throughout the store saying, "Customers, please be aware that security cameras are in operation."

[1459] Examples of online resale monitoring

[1460] 1. Server: Collects listing information via the API of an e-commerce site and detects that expensive watches of a particular brand have been listed repeatedly at abnormally low prices.

[1461] 2. Server: Flags the listing as fraudulent and sends a notification to the administrator's device saying, "A specific brand of watch has been repeatedly listed at an abnormal price."

[1462] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1463] 4. User (Administrator): Pause the item in question and investigate past sales history and other listings to identify any abnormal patterns.

[1464] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1465] In-store monitoring and announcements

[1466] Step 1: Collect video data

[1467] Server: Collects video data in real time from multiple cameras installed in the store. Specifically, the IP cameras send video data to the server using RTSP (Real Time Streaming Protocol).

[1468] Input: Real-time video data from IP cameras

[1469] Output: Raw video data stored on a server

[1470] Step 2: Analyzing the video data

[1471] Server: Analyzes collected video data using an image recognition algorithm (e.g., OpenCV or Amazon Rekognition) running on the cloud. It also analyzes the user's facial expressions using an emotion engine (e.g., Microsoft Azure's Emotion API). Detects when a customer exhibits certain suspicious behavior (e.g., hiding a product in a bag) or changes in emotion (e.g., anxiety, impatience, etc.).

[1472] Input: Raw video data

[1473] Data processing / data calculation: Analyzes video data using image recognition algorithms and emotion engines to detect suspicious behavior and changes in emotions

[1474] Output: Analysis results (presence or absence of suspicious behavior and emotional changes, location information)

[1475] Step 3: Generate alerts and notifications

[1476] Server: Based on the analysis results, if suspicious behavior or changes in emotion are detected, an alert is generated. The alert includes the details of the detected behavior, location, and changes in emotion. The generated alert is sent to the store staff's terminal.

[1477] Input: Analysis results

[1478] Data processing / data calculation: Alert information generation

[1479] Output: Alert notification sent to store staff's terminal

[1480] Step 4: Real-time announcements

[1481] Server: Uses a generative model (e.g., GPT-3) to create the content of the voice announcement. Dynamically changes the content of the announcement based on information from the emotion engine. For example, it generates a message such as "Customers, please be careful as security cameras are currently in operation," and announces it through the store's speaker system.

[1482] Input: Alert information, emotion engine analysis results

[1483] Data processing / data calculation: voice announcement generation

[1484] Output: Voice announcement played through the in-store speaker system

[1485] Step 5: On-site inspection

[1486] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and, if necessary, take appropriate action against the customer. For example, they might ask, "Excuse me, is there anything I can help you with?"

[1487] Input: Alert notification

[1488] Data processing / data calculation: On-site confirmation and response

[1489] Output: Response to suspicious behavior

[1490] Online resale monitoring

[1491] Step 1: Gather your listing information

[1492] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[1493] Input: Online Marketplace Listings API

[1494] Output: Listing data saved on the server

[1495] Step 2: Listing Analysis

[1496] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when expensive products of a particular brand are repeatedly listed at abnormally low prices.

[1497] Input: Listing information data

[1498] Data processing / data calculation: Analyzes data using pattern recognition algorithms to identify fraudulent listings

[1499] Output: Analysis results (whether or not there is fraudulent listing, seller information)

[1500] Step 3: Detect and notify fraudulent listings

[1501] Server: Based on the analysis results, the listing is flagged as fraudulent and the information is sent to the administrator's device. The notification includes information about the product, price, seller, etc.

[1502] Input: Analysis results

[1503] Data processing / data calculation: Alert information generation

[1504] Output: Alert notification sent to the administrator's device

[1505] Step 4: Send warning messages and suspend listings

[1506] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the administrator can also suspend the relevant listing.

[1507] Input: Alert notification

[1508] Data processing / data calculation: generating warning messages and suspending listings

[1509] Output: Warning message sent to seller, listing suspension status

[1510] Step 5: Further investigation

[1511] User (Administrator): After sending a warning message to the seller, conduct additional detailed checks and investigations, such as checking past transaction history and other listings to verify whether there is ongoing fraudulent activity.

[1512] Input: Seller's transaction history, past listing information

[1513] Data processing / data calculation: Analysis of transaction history and listing information

[1514] Output: Report of findings and required actions

[1515] (Application example 2)

[1516] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1517] Preventing shoplifting and vandalism is crucial for modern stores, but many current security systems have difficulty quickly and accurately detecting suspicious behavior or changes in user emotion. Furthermore, fraudulent listings on online platforms are on the rise, requiring significant effort to monitor and respond to. To solve these problems, a security system that combines more advanced technologies is needed.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1519] In this invention, the server includes a means for analyzing video data using a generative model to detect abnormal behavior and changes in facial expression, a means for generating audio announcements using the generative model and notifying the store, a means for notifying staff of an alert when abnormal behavior or changes in facial expression are detected, a means for conducting on-site inspections based on the alert notified to the staff, and a means for analyzing the user's emotional state using an emotion engine and generating an appropriate warning message. This makes it possible to quickly and accurately detect suspicious behavior and changes in facial expression and take appropriate action. It also makes it possible to detect fraudulent listings online and quickly respond to them.

[1520] A "generative model" is a type of machine learning algorithm that learns patterns from collected data and makes inferences based on new data.

[1521] "Video data" is a data format that refers to images and video information captured by cameras and other visual sensors.

[1522] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and indicates misconduct or risky behavior.

[1523] "Changes in facial expression" refers to changes in emotional state indicated by facial muscle movements, and is used to detect abnormal states such as anxiety or impatience.

[1524] "Voice announcement" is a means of conveying information through voice, and uses a generative model to automatically generate messages appropriate to the situation.

[1525] An "alert" refers to a warning message or notification issued when abnormal behavior or conditions are detected.

[1526] "Staff" refers to employees who are engaged in the operation of stores and facilities and who are responsible for safety management and customer service.

[1527] An "emotion engine" refers to technology that analyzes a user's facial expressions and behavioral data to determine their emotional state.

[1528] A "warning message" is a written or audio message that alerts a target or administrator when fraud or an abnormal condition is detected.

[1529] "Online platform" refers to a system or website for conducting transactions or exchanging information over the Internet.

[1530] "Fraudulent listing" refers to the act of selling products on online platforms with inappropriate pricing or false information.

[1531] A "pattern recognition algorithm" is a technology that automatically identifies specific patterns in data and is used to analyze image and text data.

[1532] "Listing information" refers to data on an online platform that lists product details, prices, etc.

[1533] The security system of the present invention uses a generative model and an emotion engine to monitor in-store security and fraudulent listings on online platforms. A specific system configuration and processing method for implementing the present invention are described in detail below.

[1534] System Configuration

[1535] Hardware Configuration

[1536] 1. Server:

[1537] A high-performance data analysis server (e.g., an Amazon EC2 instance).

[1538] 2. Camera:

[1539] High-resolution surveillance cameras (e.g. network cameras).

[1540] 3. Terminal:

[1541] An iOS or Android smartphone, or AR-enabled glasses (e.g., Google Glass).

[1542] Software Configuration

[1543] 1. Image Recognition Algorithm:

[1544] Uses OpenCV and TensorFlow.

[1545] 2. Emotion Engine:

[1546] It uses AWS's Amazon Rekognition and Microsoft Azure's Emotion API.

[1547] 3. Generative Model:

[1548] OpenAI's GPT-3.

[1549] Processing steps

[1550] In-store surveillance

[1551] 1. Camera footage collection and analysis:

[1552] Multiple cameras installed in the store transmit video in real time to a server, which then analyzes the video data using OpenCV and TensorFlow to detect specific behavioral patterns and changes in facial expressions.

[1553] 2. Alert generation and notification:

[1554] The server analyzes the video data using an emotion engine (AWS Rekognition) to determine the user's emotional state. If suspicious behavior or abnormal emotions are detected, an alert is generated and sent to the staff member's smartphone or smart glasses.

[1555] 3. Generate voice announcements:

[1556] If abnormal behavior or emotional changes are detected, the server uses OpenAI's GPT-3 to generate appropriate voice announcements, such as "Customers, please be aware that security cameras are currently in operation," which are broadcast throughout the store.

[1557] Online resale monitoring

[1558] 1. Listing information collection and analysis:

[1559] The server periodically collects listing information from online platforms and uses a pattern recognition algorithm to detect when a particular brand of high-priced product is repeatedly listed at an abnormally low price.

[1560] 2. Fake listing detection and notification:

[1561] The server flags the listing as fraudulent and notifies the administrator's terminal, who then checks the notification and sends a warning message to the seller.

[1562] Specific examples

[1563] 1. Camera footage analysis:

[1564] "The server receives video data from cameras in the store in real time. The received video data is analyzed using image recognition algorithms using OpenCV and TensorFlow. For example, it detects specific behavioral patterns (e.g., hiding products in a bag)."

[1565] 2. Sentiment analysis and alert generation:

[1566] "The received video data is analyzed using AWS's Amazon Rekognition to analyze the user's emotional state. If abnormal emotions such as anxiety or impatience are detected, the server generates an alert based on that information and sends it to the staff member's smartphone or smart glasses."

[1567] 3. Generate voice announcements:

[1568] "After detecting suspicious behavior or emotions, the server automatically generates an appropriate voice announcement using OpenAI's GPT-3. For example, a message such as, 'Customers, please be aware that security cameras are currently in operation,' is generated and broadcast throughout the store."

[1569] 4. Examples of prompts:

[1570] "Detect suspicious activity from the following camera footage and generate prompts to create alert messages corresponding to the activity:

[1571] Camera footage shows the customer carelessly putting a large amount of items into a bag and frequently looking back. What warning message corresponds to this behavior?

[1572] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1573] Step 1:

[1574] The server collects video data in real time from cameras installed in the store. The video data is transferred from the cameras to the server via a network. The video data is the input, and it becomes the basis for subsequent analysis processing.

[1575] Step 2:

[1576] The server analyzes the collected video data using OpenCV and TensorFlow. Specifically, it recognizes objects in the video and detects behavioral patterns. This allows it to identify abnormal behaviors and specific behavioral patterns. The output is data that includes abnormal behaviors and specific behaviors.

[1577] Step 3:

[1578] The server performs emotion analysis on the video data using AWS's Amazon Rekognition. It analyzes the facial expressions of people in the video and detects emotions such as anxiety or impatience. The input is the video data obtained in Step 2, and the output is the result of the emotion analysis.

[1579] Step 4:

[1580] If abnormal behavior or abnormal emotion is detected, the server generates an alert. The alert includes the content and location of the abnormal behavior or emotion. The alert generation includes a means to generate appropriate messages and warning content based on the analysis results. The output is alert notification data to staff.

[1581] Step 5:

[1582] The server sends the generated alert to the staff member's device (smartphone or smart glasses) via the network. The staff member's device receives the notification and displays it on its screen. The input is the generated alert data, and the output is the notification displayed on the staff member's device.

[1583] Step 6:

[1584] The server generates voice announcements using OpenAI's GPT-3. Based on the analysis of abnormal behavior and emotions, an appropriate warning message is automatically generated. The output is the generated voice announcement data.

[1585] Step 7:

[1586] The voice announcement data is broadcast through the sound system in the store. This allows everyone in the store to be alerted. The input is the voice announcement data generated in Step 6, and the output is the voice announcement from the sound system in the store.

[1587] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1588] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1590] [Fourth embodiment]

[1591] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1592] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1593] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1595] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1597] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1598] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1599] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1600] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1601] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1602] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1603] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1604] The security system of the present invention employs technology that uses generative models to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[1605] In-store monitoring and announcements

[1606] Video data collection and analysis

[1607] server

[1608] The server collects video data in real time from multiple cameras in the store. The video data received from the cameras is analyzed using image recognition algorithms. For example, trained models are used to detect specific behavioral patterns (e.g., putting items into a bag or attempting to take away a large number of items at once).

[1609] Alert generation and notification

[1610] server

[1611] When the server detects abnormal behavior, it generates an alert based on that information, which is then sent to the store staff's terminal.

[1612] Terminal

[1613] The store staff's devices receive the alert sent from the server and display a notification, such as "Suspicious activity has been detected on shelf number 2."

[1614] Real-time announcements

[1615] server

[1616] The server uses the generative model to automatically generate voice announcements and broadcast them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation."

[1617] User (store staff)

[1618] Staff can check the alert on the terminal and quickly move to the relevant area to check the situation. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[1619] Online resale monitoring

[1620] Collection and analysis of listing information

[1621] server

[1622] The server automatically collects listings from online platforms, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[1623] Fake listing detection and notification

[1624] server

[1625] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[1626] Terminal

[1627] The administrator's device receives the flag notification from the server and displays the notification. For example, the notification may say, "A particular listing may be fraudulent."

[1628] Warning messages and pauses

[1629] User (Administrator)

[1630] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, they can send a message saying, "This listing is being temporarily suspended due to the possibility of fraudulent transactions." If necessary, they can temporarily suspend the listing and conduct further investigation.

[1631] Specific examples

[1632] Specific examples of in-store monitoring and announcements

[1633] 1. The server analyzes camera footage in the store and detects suspicious behavior, such as a customer putting a large number of items into a bag at once.

[1634] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[1635] 3. Staff check the alert on their device, go to the relevant area and check the situation on site.

[1636] 4. At the same time, the server automatically generates an audio announcement and plays it throughout the store to raise awareness of crime prevention.

[1637] Examples of online resale monitoring

[1638] 1. The server collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that a specific brand of expensive watch has been repeatedly listed at abnormally low prices.

[1639] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[1640] 3. The administrator checks the notification on the device and sends a warning message to the seller stating, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1641] 4. The administrator will suspend the relevant listing and conduct further detailed checks and investigations.

[1642] The above is an example of a system of the present invention that can effectively prevent fraud both in-store and online.

[1643] The processing flow will be explained below.

[1644] In-store monitoring and announcement processing flow

[1645] Step 1:

[1646] server

[1647] The server collects video data in real time from cameras installed in the store, and the video data is immediately sent to an image recognition algorithm.

[1648] Step 2:

[1649] server

[1650] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[1651] Step 3:

[1652] server

[1653] If any abnormal behavior is detected, the server immediately generates an alert, which includes the specific details and location of the detected behavior.

[1654] Step 4:

[1655] server

[1656] The server then sends the generated alert to the store staff's device, which contains a notification such as "Suspicious activity has been detected on shelf number 2."

[1657] Step 5:

[1658] Terminal

[1659] Store staff's devices receive and display alerts from the server, allowing them to quickly identify the location of any suspicious activity.

[1660] Step 6:

[1661] User (store staff)

[1662] Staff check the alerts received on their devices, rush to the scene, confirm the actual situation, and take appropriate action.

[1663] Step 7:

[1664] server

[1665] The server uses the generative model to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[1666] Step 8:

[1667] server

[1668] The generated audio announcement is played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[1669] Online Resale Monitoring Process Flow

[1670] Step 1:

[1671] server

[1672] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[1673] Step 2:

[1674] server

[1675] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[1676] Step 3:

[1677] server

[1678] If a listing is determined to be fraudulent, the server will flag the listing, which will include details about the listing and any anomalies.

[1679] Step 4:

[1680] server

[1681] The server then sends the generated flag to the online platform administrator's terminal, which contains a notification such as "A particular listing may be fraudulent."

[1682] Step 5:

[1683] Terminal

[1684] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[1685] Step 6:

[1686] User (Administrator)

[1687] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1688] Step 7:

[1689] User (Administrator)

[1690] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[1691] In this way, the system of the present invention can efficiently implement security measures both offline and online.

[1692] Example 1

[1693] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1694] Preventing fraudulent activities is a major challenge in modern commercial facilities. Detecting and responding to shoplifting and fraudulent resale activities in real time is particularly challenging. Addressing these challenges requires building an efficient and accurate surveillance system. However, conventional systems have struggled to quickly and effectively detect and respond to these fraudulent activities. Therefore, the present invention aims to provide a new system that utilizes generative models to detect fraudulent activities in real time and respond quickly to them.

[1695] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1696] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the facility, and means for notifying workers of an alert when abnormal behavior is detected. This makes it possible to detect abnormal behavior such as shoplifting in real time and prevent fraudulent activities through audio announcements. Furthermore, the server includes means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, means for sending warning messages to sellers determined to be fraudulent, and means for suspending listings determined to be fraudulent, thereby making it possible to effectively monitor and prevent fraudulent online resale activities.

[1697] A "generative model" is a model that uses machine learning algorithms to generate information or patterns from data.

[1698] "Video data" refers to data containing visual information obtained from a camera, video recording device, or the like.

[1699] "Abnormal behavior" is behavior that deviates from normal patterns of behavior and indicates misconduct or behavior that requires attention.

[1700] A "voice announcement" is a voice message automatically generated using voice synthesis technology.

[1701] An "alert" is a warning notification sent when the system detects an abnormality.

[1702] "Worker" refers to a person in charge of monitoring and responding within a store or facility.

[1703] "Listing Information" means the information provided when a product is offered for sale on an online platform.

[1704] A "pattern recognition algorithm" is an algorithm that identifies specific patterns or features in data.

[1705] A "warning message" is a message sent to notify of irregularities or abnormalities.

[1706] "Pause" is the act of temporarily halting a particular action or process.

[1707] The system of the present invention employs techniques that leverage generative models and pattern recognition algorithms to effectively monitor and prevent fraud in-store and online.

[1708] In-store monitoring and announcements

[1709] Video data collection and analysis

[1710] The server collects video data in real time from cameras installed in the store. The cameras are installed on the ceiling and shelves, and the viewing angle and resolution are set according to the store's layout. The collected video data is analyzed using image recognition algorithms such as the YOLO model using TensorFlow. Specifically, suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once) are detected.

[1711] Alert generation and notification

[1712] When the server detects suspicious behavior, it generates an alert and sends the information in JSON format to the store staff's device. The transmission is carried out using a REST API using the HTTP protocol. The device displays the received alert as a pop-up notification, for example, saying, "Suspicious behavior has been detected on shelf number 2."

[1713] Real-time announcements

[1714] The server automatically generates a voice announcement using a generative model (e.g., Google Text-to-Speech API) and broadcasts it throughout the store. The announcement includes a warning message such as, "Security cameras are in operation, please be careful." The user (store staff) checks the alert and quickly moves to the relevant area to check the situation.

[1715] Online resale monitoring

[1716] Collection and analysis of listing information

[1717] The server periodically collects listing information from major online platforms using scraping technology. The collected information is then analyzed using pattern recognition algorithms, such as logistic regression models using scikit-learn, to identify fraudulent listings. For example, pricing that deviates significantly from normal market prices or a large number of listings in a short period of time are monitored.

[1718] Fake listing detection and notification

[1719] The server generates a "fraudulent" flag for listings that are determined to be fraudulent and sends that information in JSON format to the online platform administrator's device. The notification includes a message such as "A specific listing may be fraudulent." The device then displays the received flag as a pop-up notification.

[1720] Sending warning messages and suspending listings

[1721] The user (administrator) checks the notification and sends a warning message to the relevant seller, saying, "This listing is temporarily suspended due to the possibility of fraudulent transactions." The user also temporarily suspends the listing through the system interface and conducts additional investigation.

[1722] Specific examples

[1723] Specific examples of in-store monitoring and announcements

[1724] 1. The server analyzes camera footage in the store and detects customers putting large amounts of items into bags at once.

[1725] 2. The server generates an alert and notifies the staff member's device that "suspicious activity has been detected on shelf number 2."

[1726] 3. The user (store staff member) checks the alert on the device, goes to the relevant area and checks the situation on site.

[1727] 4. The server automatically generates voice announcements and plays them throughout the store to raise awareness of crime prevention.

[1728] Examples of online resale monitoring

[1729] 1. The server collects listing information from e-commerce sites and uses a pattern recognition algorithm to detect when expensive watches from a particular brand are repeatedly listed at abnormally low prices.

[1730] 2. The server flags the listing as fraudulent and notifies the administrator's device.

[1731] 3. The user (administrator) checks the notification on their device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1732] 4. The user (administrator) will suspend the relevant listing and conduct additional detailed checks and investigations.

[1733] The above is an embodiment of the system of the present invention, which makes it possible to effectively monitor and prevent fraudulent activities in-store and online.

[1734] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1735] In-store monitoring and announcements

[1736] Step 1: Collecting video data

[1737] The server collects video data in real time from cameras installed in the store, such as those on the ceiling or on shelves, with the viewing angle and resolution set according to the store's layout.

[1738] Input: Video data from the camera

[1739] Output: Raw video data sent to the server

[1740] Step 2: Analyzing the video data

[1741] The server analyzes the collected video data using image recognition algorithms such as the YOLO model powered by TensorFlow, and specifically detects suspicious behavioral patterns (e.g., putting items into a bag or taking away a large amount of items at once).

[1742] Input: Raw video data stored on the server

[1743] Output: Analyzed video data and suspicious behavior detection results

[1744] Step 3: Generate an alert

[1745] If the server detects any suspicious activity as a result of the analysis, it generates an alert, which includes the camera's location and timestamp.

[1746] Input: Suspicious behavior detection result

[1747] Output: Alert information (location, timestamp, etc.)

[1748] Step 4: Alert Notification

[1749] The server sends the generated alert to the store staff's terminal using the REST API with the HTTP protocol. The transmission format is JSON. The terminal displays the received alert as a pop-up notification.

[1750] Input: Alert information

[1751] Output: Notification displayed on device (e.g. "Suspicious activity detected on shelf 2")

[1752] Step 5: Generate real-time announcements

[1753] The server uses a generative model (e.g., Google Text-to-Speech API) to generate audio announcements, such as "Security cameras are active, please be careful."

[1754] Input: Alert information

[1755] Output: Generated voice announcement data

[1756] Step 6: Sending real-time announcements

[1757] The server transmits the generated voice data to a speaker system in the store, and the announcement is broadcast throughout the store.

[1758] Input: Generated voice announcement data

[1759] Output: Announcement played in store

[1760] Step 7: Staff response

[1761] The user (store staff) checks the alert on the terminal, quickly moves to the relevant area, and checks the situation on the spot. If necessary, they can also play back and check the recorded data from the surveillance camera.

[1762] Input: Alert notification displayed on the terminal

[1763] Output: On-site confirmation and response

[1764] Online resale monitoring

[1765] Step 1: Gather your listing information

[1766] The server periodically collects listing information from major online platforms using scraping technology, such as the Python library Scrapy.

[1767] Input: Online platform webpage information

[1768] Output: Listing information data (product name, price, category, seller information, etc.)

[1769] Step 2: Listing Analysis

[1770] The server analyzes the collected listing information using pattern recognition algorithms such as logistic regression models using scikit-learn to identify fraudulent listings, such as those with prices that deviate significantly from normal market prices or those with a large number of listings in a short period of time.

[1771] Input: Listing information data

[1772] Output: Identification results of fraudulent listings (listing ID, price, reason for detection, etc.)

[1773] Step 3: Generate flags

[1774] Based on the analysis results, the server generates a "Fraud" flag for fraudulent listings. This flag contains information such as the seller ID, listing ID, and the reason for detection.

[1775] Input: Result of identifying fraudulent listings

[1776] Output: Invalid flag information

[1777] Step 4: Flag Notification

[1778] The server notifies the online platform administrator of the generated fraud flag via a REST API using the HTTP protocol. The transmission format is JSON. The terminal displays the received flag as a pop-up notification.

[1779] Input: Invalid flag information

[1780] Output: Notification displayed on device (e.g. "This particular listing may be fraudulent")

[1781] Step 5: Sending a warning message

[1782] The user (administrator) checks the notification and sends a warning message to the relevant seller, such as "This listing is temporarily suspended as it may be a fraudulent transaction."

[1783] Input: Invalid flag information

[1784] Output: Warning message sent to seller

[1785] Step 6: Pause your listing

[1786] The user (administrator) temporarily suspends the relevant listing through the system interface, and takes action to stop the listing from being published in order to conduct additional investigation.

[1787] Input: Invalid flag information

[1788] Output: Paused listings

[1789] Step 7: Conduct additional research

[1790] The user (administrator) will check the details of the suspected fraudulent listing and conduct further investigation if necessary, for example, by checking the seller's past transaction data or other listed items.

[1791] Enter: Paused listings

[1792] Output: Investigation results (determination of the suitability of the listing)

[1793] The above are the specific processing steps of the system of the present invention, which enable effective monitoring and prevention of fraudulent activities in stores and online.

[1794] (Application example 1)

[1795] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1796] Conventional security systems did not provide sufficient means to prevent shoplifting and unauthorized resales occurring in stores. This resulted in a heavy burden on staff and reduced security efficiency. Furthermore, manual monitoring and response was required for online unauthorized resales, making it difficult to detect fraudulent activity early. An efficient and effective solution to these issues was needed.

[1797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1798] In this invention, the server includes means for analyzing video data using a generative model to detect abnormal behavior, means for generating audio announcements using the generative model and notifying the store, means for notifying staff of an alert when abnormal behavior is detected, means for conducting on-site inspections based on the alert notified to the staff, and means for collecting listing information and detecting fraudulent listings using a pattern recognition algorithm, thereby enabling effective monitoring and prevention of fraudulent activities both in-store and online.

[1799] A "generative model" is a model that uses machine learning algorithms to automatically learn patterns from data and perform specific tasks.

[1800] "Video data" refers to image and video information collected using a camera.

[1801] "Abnormal behavior" refers to actions or behavior that deviate from pre-established normal behavior patterns.

[1802] A "pattern recognition algorithm" is an algorithm for identifying and classifying specific patterns in data.

[1803] "Voice announcements" refer to audio notifications and guidance that are automatically generated using generative models.

[1804] An "alert" refers to a warning or notification sent to staff when the system detects abnormal behavior.

[1805] "Staff" refers to employees working in the store and those in charge of monitoring.

[1806] "Listing information" refers to detailed information about products sold on online marketplaces and e-commerce sites.

[1807] "Fraudulent listings" refer to products that are sold at prices significantly different from market value or through abnormal sales patterns.

[1808] "Warning Message" means a message sent to a seller or staff member to alert them to possible fraudulent activity.

[1809] The present invention is a security system that aims to efficiently detect and prevent fraudulent activities in stores and online using a generative AI model. Specific embodiments are described below.

[1810] In-store surveillance system

[1811] The server collects video data in real time from multiple cameras in the store. This video data is analyzed using image recognition algorithms (e.g., OpenCV or Caffe models). A trained generative model is used to detect specific behavioral patterns (e.g., putting items into a bag or trying to take away a large number of items at once).

[1812] When the server detects abnormal behavior, it immediately generates an alert and sends a notification to the store staff's device (such as a smartphone or tablet). The staff's device displays specific information, such as "Suspicious behavior has been detected on shelf number 2." The staff member checks the notification and quickly moves to the area in question to check the situation.

[1813] Furthermore, the server automatically generates voice announcements and sends them throughout the store. For example, an announcement such as, "Attention customers, please be aware that security cameras are in operation." This voice announcement will raise security awareness and is expected to have a deterrent effect on crime.

[1814] Online Resale Monitoring System

[1815] The server automatically collects listing information from major online marketplaces. The collected listing information is then analyzed using pattern recognition algorithms, analyzing information such as listing pricing and number of listings to identify listings that deviate from normal market prices and sales practices.

[1816] If the server detects fraudulent listing activity based on the analysis results, it generates a flag for the activity and sends it to the online platform administrator's device. The administrator's device displays a notification such as, "A particular listing may be fraudulent." The administrator then checks the notification and sends a warning message to the problematic seller. For example, the message might say, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1817] If necessary, the administrators will suspend the listing and conduct further detailed checks and investigations, which will enable them to quickly prevent unauthorized resale activities on the online platform.

[1818] Hardware and software used

[1819] Hardware used: security cameras, servers, smartphones, tablets

[1820] Software used: OpenCV (image recognition library), Caffe (deep learning model), REST API, push notification, Requests (HTTP request library)

[1821] Specific examples

[1822] For example, if there is ongoing fraudulent purchases or unfairly low sales of a particular product, the system will send a warning message such as, "The pricing of this product is inappropriate. We will conduct a detailed investigation."

[1823] Prompt Sentence Examples

[1824] "It uses image recognition algorithms to detect abnormal behavior in security camera footage in real time."

[1825] "We collect listing information from online platforms, detect abnormal behavioral patterns and generate alerts."

[1826] The above is a specific embodiment for carrying out the present invention, which enhances security both in-store and online, and enables rapid detection and response to fraudulent activity.

[1827] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1828] Step 1:

[1829] The server collects video data in real time from multiple cameras in the store. The video data sent from the cameras is input. The server stores the video data and prepares it for analysis.

[1830] Step 2:

[1831] The server analyzes the collected video data using image recognition algorithms (OpenCV and Caffe models). Specifically, the video data is preprocessed and input into a generative model. The generative model detects specific behavioral patterns (e.g., putting an item into a bag) and outputs the results.

[1832] Step 3:

[1833] If the server detects abnormal behavior based on the output of the generative model, it generates an alert. This alert information is output and sent to the staff's terminal. The alert contains specific information such as "Suspicious behavior has been detected on shelf number 2."

[1834] Step 4:

[1835] The terminal displays the alert received from the server. The staff member checks the notification and prepares to quickly head to the relevant area. Specifically, the staff member checks the alert content on the screen and heads to the scene.

[1836] Step 5:

[1837] When abnormal behavior is detected, the server automatically generates a voice announcement using the generative model. The generated voice announcement is output. For example, it could say, "Attention customers, please be aware that security cameras are in operation."

[1838] Step 6:

[1839] The server collects listing information from major online marketplaces, which serves as input, stores the listing information in a database, and prepares it for analysis.

[1840] Step 7:

[1841] The server analyzes the collected listing information using a pattern recognition algorithm, detecting fraudulent listings based on the listing price and number of listings, and outputs the results.

[1842] Step 8:

[1843] If the server detects fraudulent listing activity, it generates a flag and sends a notification to the online platform administrator's device, stating that "a particular listing may be fraudulent."

[1844] Step 9:

[1845] The terminal displays the flag notification received from the server. The administrator checks the notification and sends a warning message to the problematic seller. Specifically, the administrator sends a message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1846] Step 10:

[1847] The user (administrator) can suspend any listings that are deemed fraudulent as necessary, and conduct additional detailed checks and investigations. Specifically, the user selects the relevant listing from the system's administration screen and clicks the suspend button.

[1848] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1849] The security system of the present invention employs technology that combines a generative model and an emotion engine to prevent both shoplifting in offline stores and illegal online resale. The program processing of this system is described in detail below.

[1850] In-store monitoring and announcements

[1851] Video data collection and analysis

[1852] server

[1853] The server collects video data in real time from multiple cameras installed in the store. The video data received from the cameras is analyzed using image recognition algorithms and an emotion engine. For example, it detects specific behavioral patterns (e.g., hiding items in a bag, attempting to steal a large amount of merchandise) and changes in facial expressions (anxiety, impatience, etc.).

[1854] Alert generation and notification

[1855] server

[1856] When the server detects abnormal behavior or changes in emotion, it generates an alert based on that information, which includes the specific content and location of the detected behavior or emotion.

[1857] Terminal

[1858] The store staff's device receives the alert sent from the server and displays a notification, such as "Suspicious behavior and an anxious user expression were detected on shelf number 2."

[1859] Real-time announcements

[1860] server

[1861] The server uses the generative model and emotion engine to automatically generate voice announcements and broadcast them throughout the store. The content of the announcement changes dynamically depending on the user's emotional state. For example, a message such as "Customers, please be aware that security cameras are currently in operation." can be generated.

[1862] User (store staff)

[1863] Staff check the alert on their device and rush to the scene, where they can confirm the actual situation and take appropriate action. The generated announcements also raise crime prevention awareness, which is expected to have a deterrent effect on criminal acts.

[1864] Online resale monitoring

[1865] Collection and analysis of listing information

[1866] server

[1867] The server periodically collects listings from the online platform, which are then analyzed using pattern recognition algorithms to identify deviations from normal market prices and sales methods.

[1868] Fake listing detection and notification

[1869] server

[1870] If the server detects any fraudulent listing activity as a result of the analysis, it generates a flag for the activity and sends it to the terminal of the online platform administrator.

[1871] Terminal

[1872] The administrator's device receives the flag notification from the server and displays it, allowing the administrator to quickly identify any fraudulent listings.

[1873] Warning messages and pauses

[1874] User (Administrator)

[1875] The administrator checks the flag notification received on the device and sends a warning message to the problematic seller. For example, a message such as "The price setting for this item is inappropriate. We will conduct a detailed investigation" will be sent. If necessary, the listing will be suspended and additional investigation will be carried out.

[1876] Specific examples

[1877] Specific examples of in-store monitoring and announcements

[1878] 1. Server: Analyzes in-store camera footage and detects suspicious behavior and anxious expressions from users. For example, if a customer puts a large number of items into a bag at once and looks anxious.

[1879] 2. Server: Generates an alert and notifies the staff member's device that "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf 2."

[1880] 3. Terminal: Staff check the alert on the terminal and rush to the area to check the situation.

[1881] 4. Server: Using the generative model and emotion engine, an automatically generated voice announcement is played in the store, informing customers, "Customers, please be aware that security cameras are in operation."

[1882] Examples of online resale monitoring

[1883] 1. Server: Collects listing information from e-commerce sites and analyzes it using a pattern recognition algorithm. It detects that expensive watches from a specific brand have been listed repeatedly at abnormally low prices.

[1884] 2. Server: Flags the listing as fraudulent and notifies the administrator's device.

[1885] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1886] 4. User (Administrator): Pause the listing and conduct additional detailed checks and investigations.

[1887] In this way, by combining emotion engines, the present invention achieves even more accurate fraud detection and response.

[1888] The processing flow will be explained below.

[1889] In-store monitoring and announcement processing flow

[1890] Step 1:

[1891] server

[1892] The server collects video data in real time from multiple cameras installed in the store, and the video data is immediately sent to the image recognition algorithm and emotion engine.

[1893] Step 2:

[1894] server

[1895] The server analyzes the video data using image recognition algorithms, applying pre-trained models to detect specific behavioral patterns (e.g., hiding items in a bag, attempting to steal large quantities of merchandise).

[1896] Step 3:

[1897] server

[1898] The server uses an emotion engine to analyze the user's facial expressions, detecting, for example, changes in facial expression that indicate feelings of anxiety or impatience.

[1899] Step 4:

[1900] server

[1901] If any abnormal behavior or emotional changes are detected, the server immediately generates an alert, which includes the specific content and location of the detected behavior or emotion.

[1902] Step 5:

[1903] server

[1904] The server then sends the generated alert to the store staff's device, which contains a notification such as "Suspicious behavior and anxious user expression were detected on shelf number 2."

[1905] Step 6:

[1906] Terminal

[1907] Store staff's devices receive and display alerts from the server, allowing them to quickly grasp the location and circumstances of suspicious behavior.

[1908] Step 7:

[1909] User (store staff)

[1910] Staff check the alerts received on their devices, rush to the scene, check the actual situation there, and take appropriate action if necessary.

[1911] Step 8:

[1912] server

[1913] The server uses the generative model and emotion engine to automatically generate voice announcements when abnormal behavior is detected, such as "Customers please be aware that security cameras are currently in operation."

[1914] Step 9:

[1915] server

[1916] The generated audio announcement is played through speakers inside the store, which is expected to raise awareness of crime prevention and have a preventative effect.

[1917] Online Resale Monitoring Process Flow

[1918] Step 1:

[1919] server

[1920] The server periodically collects listing information from the online platform and stores the collected listing information in a database.

[1921] Step 2:

[1922] server

[1923] The server analyzes listings using pattern recognition algorithms to identify listings that deviate from normal market prices or sales methods in order to detect fraudulent listings.

[1924] Step 3:

[1925] server

[1926] The server uses an emotion engine to analyze changes in emotions from sellers' profiles and comments, detecting, for example, when a seller appears unnaturally anxious.

[1927] Step 4:

[1928] server

[1929] If a listing is deemed fraudulent or emotionally abnormal, the server flags the listing, which includes details about the listing and the abnormality.

[1930] Step 5:

[1931] server

[1932] The server then sends the generated flag to the online platform administrator's terminal, which contains a notification such as "A particular listing may be fraudulent."

[1933] Step 6:

[1934] Terminal

[1935] The administrator's device receives and displays the flag notification from the server, allowing the administrator to quickly identify any fraudulent listings.

[1936] Step 7:

[1937] User (Administrator)

[1938] The administrator will check the flag notification received on the device and send a warning message to the relevant seller, such as "The pricing for this item is inappropriate. We will conduct a detailed investigation."

[1939] Step 8:

[1940] User (Administrator)

[1941] If necessary, the administrator will take action to suspend the listing and conduct further detailed review and investigation.

[1942] In this way, by combining the emotion engine, the system of the present invention can achieve even more accurate fraud detection and response.

[1943] Example 2

[1944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1945] Conventional security systems monitor in-store shoplifting and online fraudulent resale separately, making it impossible to manage both in an integrated manner. Furthermore, there is a need for more accurate monitoring and notification that takes into account the emotional state of the user, rather than simply detecting abnormal behavior through video analysis. Furthermore, more advanced analysis and rapid notification are required to detect fraudulent listings and respond appropriately.

[1946] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1947] In this invention, the server includes means for analyzing video data using a generative model and detecting abnormal behavior and changes in emotion, means for generating audio announcements using the generative model and emotion engine and notifying the facility, and means for sending an alert to a staff member's terminal when abnormal behavior or changes in emotion are detected. This makes it possible to accurately detect fraudulent behavior within the facility and take appropriate action taking into account changes in emotion.

[1948] Additionally, for monitoring online resale, the system includes a means for collecting listing information and using pattern recognition algorithms to detect fraudulent listings, a means for sending warning messages to sellers determined to be fraudulent, and a means for suspending listings determined to be fraudulent, making it possible to quickly detect fraudulent online resale activities and take appropriate measures.

[1949] A "generative model" is an algorithm that uses machine learning and deep learning techniques to generate data such as text, images, and audio.

[1950] "Video Data" means real-time or recorded visual data collected by a camera or other image capture device.

[1951] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior, and includes behavior that is deemed suspicious for security purposes.

[1952] "Changes in emotions" refers to fluctuations in the psychological state that can be inferred from the facial expressions and attitudes of the person being observed.

[1953] An "emotion engine" is an algorithm that analyzes emotions from image and video data and identifies their state.

[1954] "Voice announcement" is an output means for generating a specific message as voice and announcing it through a speaker.

[1955] An "alert" is a notification issued when the system detects an abnormal condition or suspicious behavior, prompting a warning or action.

[1956] A "terminal" is an electronic device (such as a computer, tablet, or smartphone) that allows staff or managers to receive information and perform operations.

[1957] "Listing information" refers to data containing detailed information about products listed on online marketplaces and auction sites.

[1958] A "pattern recognition algorithm" is an algorithm that extracts regularities and patterns from data and analyzes anomalies and characteristics.

[1959] "Fraudulent listing" refers to the listing of a product that deviates from normal market prices and sales methods and violates standards and regulations.

[1960] A "warning message" is a notification sent to notify of fraudulent activity or an abnormal state, and includes content urging improvement.

[1961] "Suspension" refers to the temporary interruption or cessation of a system or operation.

[1962] The present invention is a security system for monitoring and preventing in-store and online fraud. The system uses generative models and emotion engines to detect and respond to in-store shoplifting and online fraudulent resale activities.

[1963] In-store monitoring and announcements

[1964] Server: Multiple cameras are installed in the store to collect video data in real time. Specifically, IP cameras are used to send video data to the server via RTSP (Real Time Streaming Protocol).

[1965] Server: The collected video data is analyzed in real time using image recognition algorithms (such as OpenCV or Amazon Rekognition) running on the cloud. An emotion engine such as Microsoft Azure's Emotion API is also used to analyze the user's facial expressions and detect abnormal behavior and changes in emotion. For example, the system can detect when a customer picks up an item from a shelf and puts it in a bag, or when an anxious expression appears.

[1966] Server: Generates alerts based on the detection results and sends them to store staff terminals. The alerts include the content and location of the detected behavior and changes in emotions.

[1967] Server: Creates the content of the voice announcement using a generative AI model (e.g., GPT-3). Dynamically changes the content of the announcement based on information from the emotion engine. For example, a message such as "Customers, please be aware that security cameras are in operation" is generated and announced through the in-store speaker system.

[1968] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and take action as necessary. For example, they ask the customer, "Excuse me, is there anything I can help you with?"

[1969] Online resale monitoring

[1970] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[1971] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when a high-priced product of a particular brand is repeatedly listed at an abnormally low price.

[1972] Server: Based on the analysis results, flags suspicious listings as fraudulent and notifies the administrator's device. The notification includes information about the product, price, seller, etc.

[1973] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the listing will be suspended and further investigation will be conducted.

[1974] User (Administrator): After sending a warning message, we will conduct additional detailed checks and investigations, such as checking past transaction history and other listings by the seller to verify whether there has been any ongoing fraudulent activity.

[1975] Examples of concrete examples and prompts

[1976] Specific examples of in-store monitoring and announcements

[1977] 1. Server: Collects camera footage from within the store and detects suspicious behavior and anxious expressions of customers. Example: A customer picks up multiple items from a shelf and puts them in a bag.

[1978] 2. Server: Based on the detected results, an alert is generated stating, "Suspicious behavior and a user's facial expression indicating anxiety have been detected on shelf number 2," and notified to the staff member's device.

[1979] 3. Terminal: Staff checks the alert on the terminal and rushes to the area to check the situation. Example: After staff arrives at the scene, they ask the customer, "Excuse me, is there anything I can help you with?"

[1980] 4. Server: Using the generative model and emotion engine, an audio announcement is played throughout the store saying, "Customers, please be aware that security cameras are in operation."

[1981] Examples of online resale monitoring

[1982] 1. Server: Collects listing information via the API of an e-commerce site and detects that expensive watches of a particular brand have been listed repeatedly at abnormally low prices.

[1983] 2. Server: Flags the listing as fraudulent and sends a notification to the administrator's device saying, "A specific brand of watch has been repeatedly listed at an abnormal price."

[1984] 3. Device: The administrator checks the notification on the device and sends a warning message to the seller saying, "The pricing for this item is inappropriate. We will conduct a detailed review."

[1985] 4. User (Administrator): Pause the item in question and investigate past sales history and other listings to identify any abnormal patterns.

[1986] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1987] In-store monitoring and announcements

[1988] Step 1: Collect video data

[1989] Server: Collects video data in real time from multiple cameras installed in the store. Specifically, the IP cameras send video data to the server using RTSP (Real Time Streaming Protocol).

[1990] Input: Real-time video data from IP cameras

[1991] Output: Raw video data stored on a server

[1992] Step 2: Analyzing the video data

[1993] Server: Analyzes collected video data using an image recognition algorithm (e.g., OpenCV or Amazon Rekognition) running on the cloud. It also analyzes the user's facial expressions using an emotion engine (e.g., Microsoft Azure's Emotion API). Detects when a customer exhibits certain suspicious behavior (e.g., hiding a product in a bag) or changes in emotion (e.g., anxiety, impatience, etc.).

[1994] Input: Raw video data

[1995] Data processing / data calculation: Analyzes video data using image recognition algorithms and emotion engines to detect suspicious behavior and changes in emotions

[1996] Output: Analysis results (presence or absence of suspicious behavior and emotional changes, location information)

[1997] Step 3: Generate alerts and notifications

[1998] Server: Based on the analysis results, if suspicious behavior or changes in emotion are detected, an alert is generated. The alert includes the details of the detected behavior, location, and changes in emotion. The generated alert is sent to the store staff's terminal.

[1999] Input: Analysis results

[2000] Data processing / data calculation: Alert information generation

[2001] Output: Alert notification sent to store staff's terminal

[2002] Step 4: Real-time announcements

[2003] Server: Uses a generative model (e.g., GPT-3) to create the content of the voice announcement. Dynamically changes the content of the announcement based on information from the emotion engine. For example, it generates a message such as "Customers, please be careful as security cameras are currently in operation," and announces it through the store's speaker system.

[2004] Input: Alert information, emotion engine analysis results

[2005] Data processing / data calculation: voice announcement generation

[2006] Output: Voice announcement played through the in-store speaker system

[2007] Step 5: On-site inspection

[2008] User (store staff): The staff member checks the alert on the terminal and rushes to the designated shelf or area. They check for suspicious behavior and, if necessary, take appropriate action against the customer. For example, they might ask, "Excuse me, is there anything I can help you with?"

[2009] Input: Alert notification

[2010] Data processing / data calculation: On-site confirmation and response

[2011] Output: Response to suspicious behavior

[2012] Online resale monitoring

[2013] Step 1: Gather your listing information

[2014] Server: Using the API of a specific online marketplace (e.g., eBay or Amazon), the server periodically collects listing information, which is then stored in a database on the server.

[2015] Input: Online Marketplace Listings API

[2016] Output: Listing data saved on the server

[2017] Step 2: Listing Analysis

[2018] Server: Analyzes the collected listing information using a pattern recognition algorithm (e.g., TensorFlow) to identify listings that deviate from normal market prices or sales methods. For example, it detects when expensive products of a particular brand are repeatedly listed at abnormally low prices.

[2019] Input: Listing information data

[2020] Data processing / data calculation: Analyzes data using pattern recognition algorithms to identify fraudulent listings

[2021] Output: Analysis results (whether or not there is fraudulent listing, seller information)

[2022] Step 3: Detect and notify fraudulent listings

[2023] Server: Based on the analysis results, the listing is flagged as fraudulent and the information is sent to the administrator's device. The notification includes information about the product, price, seller, etc.

[2024] Input: Analysis results

[2025] Data processing / data calculation: Alert information generation

[2026] Output: Alert notification sent to the administrator's device

[2027] Step 4: Send warning messages and suspend listings

[2028] User (Administrator): The administrator checks the notification on their device and sends a warning message to the fraudulent seller. For example, the message might say, "The price setting for this item is inappropriate. We will conduct a detailed investigation." If necessary, the administrator can also suspend the relevant listing.

[2029] Input: Alert notification

[2030] Data processing / data calculation: generating warning messages and suspending listings

[2031] Output: Warning message sent to seller, listing suspension status

[2032] Step 5: Further investigation

[2033] User (Administrator): After sending a warning message to the seller, conduct additional detailed checks and investigations, such as checking past transaction history and other listings to verify whether there is ongoing fraudulent activity.

[2034] Input: Seller's transaction history, past listing information

[2035] Data processing / data calculation: Analysis of transaction history and listing information

[2036] Output: Report of findings and required actions

[2037] (Application example 2)

[2038] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2039] Preventing shoplifting and vandalism is crucial for modern stores, but many current security systems have difficulty quickly and accurately detecting suspicious behavior or changes in user emotion. Furthermore, fraudulent listings on online platforms are on the rise, requiring significant effort to monitor and respond to. To solve these problems, a security system that combines more advanced technologies is needed.

[2040] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2041] In this invention, the server includes a means for analyzing video data using a generative model to detect abnormal behavior and changes in facial expression, a means for generating audio announcements using the generative model and notifying the store, a means for notifying staff of an alert when abnormal behavior or changes in facial expression are detected, a means for conducting on-site inspections based on the alert notified to the staff, and a means for analyzing the user's emotional state using an emotion engine and generating an appropriate warning message. This makes it possible to quickly and accurately detect suspicious behavior and changes in facial expression and take appropriate action. It also makes it possible to detect fraudulent listings online and quickly respond to them.

[2042] A "generative model" is a type of machine learning algorithm that learns patterns from collected data and makes inferences based on new data.

[2043] "Video data" is a data format that refers to images and video information captured by cameras and other visual sensors.

[2044] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and indicates misconduct or risky behavior.

[2045] "Changes in facial expression" refers to changes in emotional state indicated by facial muscle movements, and is used to detect abnormal states such as anxiety or impatience.

[2046] "Voice announcement" is a means of conveying information through voice, and uses a generative model to automatically generate messages appropriate to the situation.

[2047] An "alert" refers to a warning message or notification issued when abnormal behavior or conditions are detected.

[2048] "Staff" refers to employees who are engaged in the operation of stores and facilities and who are responsible for safety management and customer service.

[2049] An "emotion engine" refers to technology that analyzes a user's facial expressions and behavioral data to determine their emotional state.

[2050] A "warning message" is a written or audio message that alerts a target or administrator when fraud or an abnormal condition is detected.

[2051] "Online platform" refers to a system or website for conducting transactions or exchanging information over the Internet.

[2052] "Fraudulent listing" refers to the act of selling products on online platforms with inappropriate pricing or false information.

[2053] A "pattern recognition algorithm" is a technology that automatically identifies specific patterns in data and is used to analyze image and text data.

[2054] "Listing information" refers to data on an online platform that lists product details, prices, etc.

[2055] The security system of the present invention uses a generative model and an emotion engine to monitor in-store security and fraudulent listings on online platforms. A specific system configuration and processing method for implementing the present invention are described in detail below.

[2056] System Configuration

[2057] Hardware Configuration

[2058] 1. Server:

[2059] A high-performance data analysis server (e.g., an Amazon EC2 instance).

[2060] 2. Camera:

[2061] High-resolution surveillance cameras (e.g. network cameras).

[2062] 3. Terminal:

[2063] An iOS or Android smartphone, or AR-enabled glasses (e.g., Google Glass).

[2064] Software Configuration

[2065] 1. Image Recognition Algorithm:

[2066] Uses OpenCV and TensorFlow.

[2067] 2. Emotion Engine:

[2068] It uses AWS's Amazon Rekognition and Microsoft Azure's Emotion API.

[2069] 3. Generative Model:

[2070] OpenAI's GPT-3.

[2071] Processing steps

[2072] In-store surveillance

[2073] 1. Camera footage collection and analysis:

[2074] Multiple cameras installed in the store transmit video in real time to a server, which then analyzes the video data using OpenCV and TensorFlow to detect specific behavioral patterns and changes in facial expressions.

[2075] 2. Alert generation and notification:

[2076] The server analyzes the video data using an emotion engine (AWS Rekognition) to determine the user's emotional state. If suspicious behavior or abnormal emotions are detected, an alert is generated and sent to the staff member's smartphone or smart glasses.

[2077] 3. Generate voice announcements:

[2078] If abnormal behavior or emotional changes are detected, the server uses OpenAI's GPT-3 to generate appropriate voice announcements, such as "Customers, please be aware that security cameras are currently in operation," which are broadcast throughout the store.

[2079] Online resale monitoring

[2080] 1. Listing information collection and analysis:

[2081] The server periodically collects listing information from online platforms and uses a pattern recognition algorithm to detect when a particular brand of high-priced product is repeatedly listed at an abnormally low price.

[2082] 2. Fake listing detection and notification:

[2083] The server flags the listing as fraudulent and notifies the administrator's terminal, who then checks the notification and sends a warning message to the seller.

[2084] Specific examples

[2085] 1. Camera footage analysis:

[2086] "The server receives video data from cameras in the store in real time. The received video data is analyzed using image recognition algorithms using OpenCV and TensorFlow. For example, it detects specific behavioral patterns (e.g., hiding products in a bag)."

[2087] 2. Sentiment analysis and alert generation:

[2088] "The received video data is analyzed using AWS's Amazon Rekognition to analyze the user's emotional state. If abnormal emotions such as anxiety or impatience are detected, the server generates an alert based on that information and sends it to the staff member's smartphone or smart glasses."

[2089] 3. Generate voice announcements:

[2090] "After detecting suspicious behavior or emotions, the server automatically generates an appropriate voice announcement using OpenAI's GPT-3. For example, a message such as, 'Customers, please be aware that security cameras are currently in operation,' is generated and broadcast throughout the store."

[2091] 4. Examples of prompts:

[2092] "Detect suspicious activity from the following camera footage and generate prompts to create alert messages corresponding to the activity:

[2093] Camera footage shows the customer carelessly putting a large amount of items into a bag and frequently looking back. What warning message corresponds to this behavior?

[2094] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2095] Step 1:

[2096] The server collects video data in real time from cameras installed in the store. The video data is transferred from the cameras to the server via a network. The video data is the input, and it becomes the basis for subsequent analysis processing.

[2097] Step 2:

[2098] The server analyzes the collected video data using OpenCV and TensorFlow. Specifically, it recognizes objects in the video and detects behavioral patterns. This allows it to identify abnormal behaviors and specific behavioral patterns. The output is data that includes abnormal behaviors and specific behaviors.

[2099] Step 3:

[2100] The server performs emotion analysis on the video data using AWS's Amazon Rekognition. It analyzes the facial expressions of people in the video and detects emotions such as anxiety or impatience. The input is the video data obtained in Step 2, and the output is the result of the emotion analysis.

[2101] Step 4:

[2102] If abnormal behavior or abnormal emotion is detected, the server generates an alert. The alert includes the content and location of the abnormal behavior or emotion. The alert generation includes a means to generate appropriate messages and warning content based on the analysis results. The output is alert notification data to staff.

[2103] Step 5:

[2104] The server sends the generated alert to the staff member's device (smartphone or smart glasses) via the network. The staff member's device receives the notification and displays it on its screen. The input is the generated alert data, and the output is the notification displayed on the staff member's device.

[2105] Step 6:

[2106] The server generates voice announcements using OpenAI's GPT-3. Based on the analysis of abnormal behavior and emotions, an appropriate warning message is automatically generated. The output is the generated voice announcement data.

[2107] Step 7:

[2108] The voice announcement data is broadcast through the sound system in the store. This allows everyone in the store to be alerted. The input is the voice announcement data generated in Step 6, and the output is the voice announcement from the sound system in the store.

[2109] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2110] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2111] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2112] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2113] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2114] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2115] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2116] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2117] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2118] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2119] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2120] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2121] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 ma...

Claims

1. A means for analyzing video data using a generative model and detecting abnormal behavior; a means for generating a voice announcement using the generative model and notifying the announcement in the store; A means of alerting staff when abnormal behavior is detected; A means of conducting on-site inspections based on alerts notified to staff; A system including:

2. 10. The system of claim 1, further comprising means for using pattern recognition algorithms in analyzing abnormal behavior.

3. a means of collecting listing information and using pattern recognition algorithms to detect fraudulent listings; A means for sending a warning message to sellers who are determined to be fraudulent; A means to suspend listings that are deemed fraudulent, and The system of claim 1 , comprising:

Images