System and method for reporting using photo

Abstract

A system and method for reporting cases using photographs are provided. The method includes the following steps: a photograph identification module uses deep learning technology to identify whether a photograph is a forgery; a photograph analysis module uses an artificial intelligence model to analyze the objects in the photograph; a photograph location acquisition module obtains the photograph's shooting location; a context recognition module uses context recognition technology to identify the context corresponding to the photograph; a case classification module obtains the case category corresponding to the photograph; and a case processing module transmits a case notification message to the corresponding local authority at the photograph location.

Description

Technical Field

[0001] This invention relates to a system and method for reporting cases using photographs. Prior Technology

[0002] When a person reporting an emergency needs urgent assistance from police, rescue personnel, or medical staff, they typically report it via voice, internet, or text message. However, if the person reporting is in a dangerous environment (such as a noisy place, filled with unknown gases or smoke, or in a hazardous situation), and cannot clearly explain the situation verbally or by typing, it will delay the opportunity to report the incident. Summary of the Invention

[0003] The system for reporting cases using photographs according to this invention includes a photograph identification module, a photograph analysis module, a shooting location acquisition module, a context recognition module, a case classification module, and a case processing module. The photograph identification module uses deep learning technology to identify whether a photograph is forged. The photograph analysis module uses an artificial intelligence model to analyze the objects in the photograph. The shooting location acquisition module obtains the shooting location of the photograph. The context recognition module uses context recognition technology to identify the context corresponding to the photograph. The case classification module obtains the case category corresponding to the photograph. The case processing module transmits a case notification message to the corresponding local authority at the shooting location, wherein the case notification message includes the shooting location, context, and case category.

[0004] The method for reporting cases using photos according to the present invention includes the following steps: a photo identification module uses deep learning technology to identify whether a photo is a forgery; a photo analysis module uses an artificial intelligence model to analyze the objects in the photo; a shooting location acquisition module obtains the shooting location of the photo; a context recognition module uses context recognition technology to identify the context corresponding to the photo; a case classification module obtains the case category corresponding to the photo; and a case processing module transmits a case notification message to the corresponding local agency device at the shooting location, wherein the case notification message includes the shooting location, context, and case category. Simple Explanation of the Diagram

[0005] Figure 1 is a schematic diagram illustrating a system for reporting cases using photographs according to an embodiment of the present invention. Figure 2 is a flowchart illustrating a method for reporting cases using photographs according to an embodiment of the present invention. Figure 3 is a schematic diagram of the operation of the system shown in Figure 1. Figure 4 is another schematic diagram of the operation of the system shown in Figure 1. Implementation

[0006] Figure 1 is a schematic diagram illustrating a system 100 for case reporting using photographs according to an embodiment of the present invention. In this embodiment, system 100 may include a photograph identification module 102, a photograph analysis module 103, a shooting location acquisition module 104, a context recognition module 105, a case classification module 106, and a case processing module 107. In one embodiment, system 100 may include a historical photograph database 20. In one embodiment, system 100 may include a GPS location database 41. In one embodiment, system 100 may include a location photograph database 42. In one embodiment, system 100 may include a case category database 60. In one embodiment, system 100 may include a case photograph database 51.

[0007] In one embodiment, system 100 may include a message receiving module 101. Message receiving module 101 may receive a photo-containing text message sent from a notifying device. The text message may include a photo. The photo may be, for example, a digital photo. The text message may be, for example, a multimedia text message; however, the invention is not limited thereto.

[0008] Figure 2 is a flowchart illustrating a method for reporting cases using photographs according to an embodiment of the present invention, wherein the method can be implemented by the system 100 shown in Figure 1. Please refer to Figures 1 and 2 simultaneously.

[0009] In step S201, the photo authentication module 102 can utilize deep learning technology to identify whether a photo is a forgery. In one embodiment, the photo authentication module 102 can utilize deep learning technology to identify differences between a real photo and a forgery. These differences may include lighting, shadows, proportions, and text. Specifically, the photo authentication module 102 can use deep learning technology to distinguish differences between real photos and forgery samples, such as lighting, shadows, proportions, and text. Therefore, the photo authentication module 102 can detect whether a photo has traces of fabrication or identify inconsistencies between real and forgery photos to confirm the authenticity of the photo. In another embodiment, the historical photo database 20 can store historical photos. The photo authentication module 102 can utilize reverse image search technology to identify whether a photo is a historical photo. Specifically, the photo authentication module 102 can utilize reverse image search technology to compare with historical photos. Therefore, the photo authentication module 102 can identify whether a photo is a historical photo.

[0010] In step S202, the photo analysis module 103 can utilize an artificial intelligence model to analyze the objects in the photo. Specifically, the photo analysis module 103 can use a pre-trained artificial intelligence model to analyze the photo content. The artificial intelligence model can be, for example, any artificial intelligence / machine learning model capable of recognizing objects, scenes, and contexts in a photo, such as a convolutional neural network (CNN), a long short-term memory network (LSTM), or other deep learning models. The artificial intelligence model can be used to analyze the content displayed in the photo to obtain relevant information. More specifically, the artificial intelligence model can be pre-trained using a dataset containing millions of photos, which may include photo labels, including the photo's shooting location and the context depicted in the photo.

[0011] In step S203, the shooting location acquisition module 104 obtains the shooting location of the photo. In one embodiment, the GPS location database 41 stores shooting locations corresponding to GPS information. When the shooting location acquisition module 104 obtains GPS information from the photo, it can use the GPS location database 41 to obtain the shooting location of the photo. Specifically, the photo may be taken by a GPS-enabled reporting device (e.g., a smartphone). Therefore, the photo may include GPS information of the shooting location. The shooting location acquisition module 104 can use the GPS location database 41 to locate / determine the shooting location of the photo. Alternatively, the shooting location acquisition module 104 can convert the GPS information of the shooting location into latitude and longitude format, and then obtain the shooting location address corresponding to this GPS information through a reverse geocoding service (such as the Google Maps API).

[0012] On the other hand, when the shooting location acquisition module 104 does not obtain GPS information from the photo, it can use an artificial intelligence model, a location photo database 42, and shooting location auxiliary information in the photo to obtain the shooting location of the photo. In one embodiment, the shooting location auxiliary information may include unique natural landscapes, landmarks, store names, buildings, street signs, and signs. Specifically, the location photo database 42 can map objects, events / situations, and other real-world phenomena to specific geographical areas marked by latitude and longitude coordinates. In addition, the location photo database 42 can combine location information with the characteristics or attributes of a dataset of related things within a specified period, such as the shooting location of the photo. When the shooting location acquisition module 104 does not obtain GPS information from the photo, it can use a pre-trained artificial intelligence model to obtain the shooting location based on some shooting location auxiliary information (such as landmarks, store names, buildings, street signs, and / or signs) contained in the photo that can indicate the shooting location. For example, the shooting location acquisition module 104 can obtain the shooting location by recognizing landmarks. In detail, the location acquisition module 104 can pre-train a convolutional neural network model to identify landmarks in a photograph. After the location acquisition module 104 inputs the photograph into the model, the model outputs a list of landmarks contained in the photograph. Then, the location acquisition module 104 can query a map service to obtain the geographical locations of these landmarks, thereby determining the photograph's shooting location.

[0013] Taking a photo with the famous Taipei 101 hotel in the background as an example, the shooting location module 104 can identify the Taipei 101 hotel in the photo by training a convolutional neural network model. The detailed method is as follows:

[0014] (1) Collect a dataset containing photos of Taipei 101 Hotel and corresponding location information. For example, photos of Taipei 101 Hotel can be obtained through online searches, by taking photos with a camera, or by downloading photos of Taipei 101 Hotel from public databases. The dataset should cover photos of Taipei 101 Hotel from different angles, under different lighting and weather conditions. The shooting location acquisition module 104 can preprocess the photos, such as resizing, cropping, and normalizing.

[0015] (2) Choose a suitable architecture for the convolutional neural network model, such as VGG16 or ResNet50. Add a fully connected layer on top of the model to output the predicted probability of Taipei 101 Hotel.

[0016] (3) Train the CNN model using the selected dataset. In addition, a loss function can be used to measure the model's prediction error. A loss function such as cross-entropy can be used. Furthermore, optimization algorithms (such as gradient descent) can be used to update the model parameters.

[0017] (4) Evaluate the model's performance using the test dataset. Calculate metrics such as precision, recall, and F1 score. Specifically, "precision" is the probability of correctly predicting a photo. "Recall" is the probability that all actual photos of Taipei 101 are correctly identified.

[0018] (5) Use the geographic coordinates of Taipei 101 Hotel as the geographic location. The geographic coordinates of Taipei 101 Hotel are: latitude: 25.049690, longitude: 121.947197. Output the predicted location of Taipei 101 Hotel in the photo.

[0019] Based on this, the shooting location acquisition module 104 can use landmarks in the photo to output the predicted shooting location using the following steps: Input the image photo into a trained convolutional neural network model. Obtain the model's prediction probability. If the prediction probability is greater than a certain threshold, output the predicted shooting location of the Taipei 101 hotel in the photo.

[0020] Please refer to Figure 2. In step S204, the context recognition module 105 can use context recognition technology to identify the context corresponding to the photograph. Specifically, the context recognition module 105 can analyze the relationships between objects, scenes, and people in the photograph to determine the context displayed in the photograph. The context recognition module 105 can refer to pre-stored case photographs in the case photograph database 51 to identify the context corresponding to the photograph. For example, suppose the photograph is a photograph of a traffic accident scene. The context recognition module 105 can use context recognition technology / photo analysis technology to quickly extract key information from the traffic accident scene, such as vehicle collision marks and pedestrian injuries.

[0021] Taking a photo of a car and a motorcycle colliding as an example, the context recognition module 105 can identify / infer the context in the photo by training an AI model. The detailed process is as follows:

[0022] (1) Collect a dataset containing photos of car accidents and corresponding descriptions of the accidents. The dataset covers car accident photos of various accident types, vehicle types, weather conditions, shooting angles, scenes, and severity. The data and sources for the context recognition module 105 can be from searching for car accident photos and descriptions on the Internet, obtaining data from transportation departments or insurance companies, or obtaining data through manual labeling of car accident photos and descriptions. Next, the context recognition module 105 can preprocess the images, such as resizing, cropping, and normalizing.

[0023] (2) Choose a suitable AI model architecture, such as a deep learning model or a traditional machine learning model. The model should be able to extract features from the image and describe the car accident situation based on these features.

[0024] (3) Train the AI ​​model using the selected dataset. Use a loss function, such as cross-entropy, to measure the model's prediction error. Use an optimization algorithm, such as gradient descent, to update the model parameters.

[0025] (4) Use test datasets to evaluate the performance of the model by calculating metrics such as accuracy, recall and F1 score.

[0026] (5) Input the photos of the car accident into the trained AI model. The model can then obtain a description (context) of the car accident situation. For a photo containing a collision between a passenger car and a motorcycle, the model can correctly describe the car accident situation, including the type of vehicle, the circumstances of the collision, the injuries of the occupants of the vehicles, and the injuries of the pedestrians.

[0027] Please refer to Figure 2. In step S205, the case classification module 106 can obtain the case category corresponding to the photograph. In one embodiment, the case category database 60 can store case type rule data and historical case records. The case classification module 106 can use the case type rule data and historical case records to obtain the case category corresponding to the photograph. Specifically, the case category database 60 can be used to store pre-collected case category information and can also be used to store case type rule data. The case classification module 106 can analyze the case category corresponding to the photograph based on the case type rule data and historical case records stored in the case category database 60. The case category database 60 can define detailed rules for what type of case a case may belong to. Case categories include, for example, disaster cases (fire, landslide, chemical substances, etc.), traffic cases (major car accidents, traffic accidents, etc.), criminal cases (fighting, smuggling, serious injury, kidnapping for ransom, etc.), etc., but the present invention is not limited to these.

[0028] In step S206, the case processing module 107 can transmit a case notification message to the local authority device corresponding to the shooting location. The case notification message includes the shooting location (crime location), context (case description), and case type. In one embodiment, the local authority device corresponding to the shooting location may include the local police station or fire station device; however, the invention is not limited thereto. After receiving the case notification message via the local authority device, the local police station can immediately dispatch officers to handle and file the case.

[0029] Figure 3 is a schematic diagram of the operation of the system 100 shown in Figure 1. Figure 4 is another schematic diagram of the operation of the system 100 shown in Figure 1. After receiving a text message including a photo from the reporting device, the system 100 can identify whether the photo is a forgery, analyze the objects in the photo, obtain the location where the photo was taken, identify the context corresponding to the photo, and obtain the case category corresponding to the photo. Then, the case processing module 107 of the system 100 can transmit a case notification message to the corresponding agency device 300 at the location where the photo was taken, wherein the case notification message may include the location where the photo was taken (the address of the incident), the context (case description), and the case category.

[0030] In one embodiment, when the system 100 is used for occupational safety emergency notification within a manufacturing plant, the system 100 may not need to execute step S201 shown in FIG2.

[0031] In one embodiment, when system 100 is used for occupational safety emergency notification, the occupational safety department device can obtain the location of the accident (incident address) where the photo was taken, and identify the accident situation (case description) and accident type in the photo. Then, the case notification message is transmitted to the corresponding local agency device at the location where the photo was taken.

[0032] In one embodiment, the modules described above may be software and / or firmware code executed by a processor.

[0033] In summary, the system and method for reporting cases using photographs of the present invention can automatically identify whether a photograph is forged, analyze the objects in the photograph, obtain the photograph's shooting location, and identify the context of the photograph. Then, it can transmit the case report message to the corresponding local authority at the photograph's location. In this way, the present invention allows the reporter to report cases more promptly and improves the efficiency of local authorities in handling cases.

[0034] 100: The system for reporting cases using photos 101: Message Receiving Module 102: Photo Authentication Module 103: Photo Analysis Module 104: Camera position acquisition module 105: Context Recognition Module 106: Case Classification Module 107: Case Processing Module 20: Historical Photo Archive 41: GPS Location Database 42: Location Photo Database 51: Case Photo Database 60: Case Category Database S201, S202, S203, S204, S205, S206: Steps 300: Local corresponding mechanisms and devices at the shooting location

Claims

1. A system for reporting cases using photographs, comprising: The photo identification module uses deep learning technology to identify whether the photo is a fake photo; The photo analysis module uses artificial intelligence models to analyze the objects in the photo; The system comprises the following modules: a shooting location acquisition module, which acquires the shooting location of the photograph; a context recognition module, which uses context recognition technology to identify the context corresponding to the photograph, wherein the context recognition module uses the context recognition technology to analyze the relationship between the objects, scenes, and people to identify the context corresponding to the photograph; a case classification module, which acquires the case category corresponding to the photograph; and a case processing module, which transmits a case notification message to the corresponding local agency device at the shooting location, wherein the case notification message includes the shooting location, the context, and the case category. The system further includes a location photograph database, wherein when the shooting location acquisition module does not acquire GPS information from the photograph, the shooting location acquisition module uses the artificial intelligence model, the location photograph database, and shooting location auxiliary information in the photograph to acquire the shooting location of the photograph, wherein the location photograph database stores mapping relationships between the objects, the context, and real phenomena to specific geographical areas marked by latitude and longitude coordinates. The shooting location acquisition module inputs the photo into the artificial intelligence model to output a list of shooting location auxiliary information contained in the photo. The shooting location acquisition module queries the location photo database or map service to obtain the geographical location corresponding to the shooting location auxiliary information, thereby obtaining the shooting location of the photo. The shooting location auxiliary information includes unique natural landscapes, landmarks, store names, buildings, street signs, and signs.

2. The system as claimed in claim 1, wherein the photo identification module utilizes the deep learning technology to identify differences between the photo and the forged photo, wherein the differences include lighting, shadows, proportions, and text.

3. The system as described in claim 1 further includes a historical photo database, wherein the historical photo database stores historical photos, and wherein the photo identification module further utilizes reverse image search technology to identify whether the photo is a historical photo.

4. The system as described in claim 1 further includes a GPS location database, wherein the GPS location database stores the shooting location corresponding to the GPS information, wherein when the shooting location acquisition module acquires the GPS information in the photograph, the shooting location acquisition module uses the GPS location database to acquire the shooting location of the photograph.

5. The system as claimed in claim 1 further includes a message receiving module, wherein the message receiving module receives a text message sent from a notifying device, wherein the text message includes the photograph.

6. The system as described in claim 1 further includes a case category database, wherein the case category database stores case type rule data and historical case records, wherein the case classification module uses the case type rule data and the historical case records to obtain the case category corresponding to the photograph.

7. The system as claimed in claim 1, wherein the local corresponding agency device at the shooting location includes a local police device at the shooting location.

8. A method for reporting cases using photographs, suitable for a system including a photograph identification module, a photograph analysis module, a shooting location acquisition module, a context recognition module, a case classification module, and a case processing module, wherein the method includes the following steps: the photograph identification module uses deep learning technology to identify whether the photograph is a forged photograph; the photograph analysis module uses an artificial intelligence model to analyze the objects in the photograph; the shooting location acquisition module obtains the shooting location of the photograph; the context recognition module uses context recognition technology to identify the context corresponding to the photograph, wherein the context recognition module uses the context recognition technology to analyze the relationship between the objects, the scene, and the people to identify the context corresponding to the photograph; the case classification module obtains the case category corresponding to the photograph; and the case processing module transmits a case notification message to the corresponding local authority at the shooting location, wherein the case notification message includes the shooting location, the context, and the case category. The system further includes a location photo database, which stores mapping relationships between objects, situations, and real phenomena and specific geographical areas indicated by latitude and longitude coordinates. The method further includes the following steps: when the shooting location acquisition module does not obtain GPS information from the photo, the shooting location acquisition module uses the artificial intelligence model, the location photo database, and shooting location auxiliary information in the photo to obtain the shooting location of the photo; the shooting location acquisition module inputs the photo into the artificial intelligence model to output a list of shooting location auxiliary information contained in the photo; and the shooting location acquisition module queries the location photo database or map service to obtain the geographical location corresponding to the shooting location auxiliary information, thereby obtaining the shooting location of the photo. The shooting location auxiliary information includes unique natural landscapes, landmarks, store names, buildings, street signs, and signs.

Images