AI powered self-learning system for emergencies recognition, reaction and crimes prevention

Abstract

An emergency response system and method comprises a smartphone emergency call being initiated though a preinstalled Internet messenger by the use of hardware and / or software which starts the recording of audio, photo and video at the scene of emergency immediately after activation. The phone transmits this audio / video with the location and ID information (step 1) through Internet to the paired Internet messenger of the disclosed AI powered self-learning system for emergencies recognition (step 2), reaction and crime prevention which can recognise the type of emergency and can transmit the processed data from the scene to the most suitable team(s) of first responders with the proposed optimal route to reach the emergency and the possible recommended algorithm of actions on the scene (step 3). After the emergency is resolved the system adds audio / video records of it to the database for the detection and recognising of an emergency next time (step 4). The system can work fully autonomously but is also in a constant communication with and under the control of “999” emergency operators.

Description

First responders’ system of communications and their reactions during emergency are crucial to save lives and to provide help. Existing in the UK and the similar systems in other countries can be improved in several ways -starting from an initiating an emergency calls , a recognition of an emergency and different reactions on it. The need for a new emergency services communication system is widely acknowledged and the work in this direction is now conducting in many countries including the UK. Thus in 2023 The UK Home Office (HO) paid Motorola £27m to pull out of a contract to provide a new emergency services communications system, according to the National Audit Office (NAO)...work on ESN began in 2015 when the HO signed contracts with Motorola and the mobile operator EE to develop a new system to replace “Airwave”, the current communications system used by fire, police and ambulance services. Overall £2bn has been spent on the project so far, said the report, see: https: / / www.nao.org.uk / press-releases / progress-with-delivering-the-emergency-services-network / Home Office said “The Emergency Services Network (ESN) will provide first responders with better technology and faster access to life-saving data in emergency situations, helping to keep the public safe...we are committed to addressing the delays and working closely with our partners to provide better value for money for the taxpayer, following Motorola’s decision to leave the programme...” see: https: / / www.gov.uk / government / publications / the-emergency-services-mobile-communications-programme / emergency-services-network In the US the similar improvement of the ESN program “FirstNet” in the stage of development since 2012 with the purpose of operate a public safety broadband network with a single platform for law enforcement, firefighters, paramedics and other public safety officials but still far from been workable see: https: / / firstnet.gov / and https: / / forums.att.com / conversations / wireless-account / switching-to-firstnet-issuesproblems / 6472b58bc3b1ef2541c72442 In 2022 only one police force in the UK is meeting a target to answer 90% of 999 calls is under the 10 seconds, Government data has shown. It's worth mentioning that 10 seconds when somebody, for example, faces a knife attack - means the difference between life and death. Also, one must bear in mind that the target of 10 seconds is only to answer the call and not to start any reaction or the first responders activity. Thus the need to improve the first responders system of communication, data flows and the algorithm of actions do exist and quite urgent. Another aim of this invention is to help in crimes prevention through the deterrence because of the inevitability for the wrongdoers to be caught on camera and be reported to a law enforcements during a crime or an attempt to commit a crime. The system needn’t to wait the answer from 999 operator or any actions (after triggering the emergency knob or the software) from the victim to start the recording the situation including all persons around, to install the fully autonomous connection and to share data with police . The general public’s awareness that in every pocket can be a “policeman’s eye” which can be triggered almost instantly by any person nearby (even during the last seconds of a life of the victim) must be a serious deterrence measure for any criminals. The knowledge by the attackers that his actions can be transmitted directly to the police station live should prevent substantially - for example - knife crimes and other types of violent behaviour. The system also makes it easier to investigate and to prosecute the recorded on video directly by a victim or a witness crimes. The reaction time during an emergency situation is crucial and consists of the following main components: 1. Time necessary for a victim or a witness to unlock the phone and to call (dial) 999 (911, 112 etc.) commutator. 2. Time for an operator to answer the call. 3. Time to explain “what is your emergency” (in case a victim still able to speak to the moment till the operator’s answer). 4. Time for the recognition and classification of the emergency by an “999” operator which will be based on the words of a victim who can be in an inadequate state or under influence or wounded or in a distress so the real picture on the scene of emergency can be quite differ from the victim’s and / or witnesses’ words. 5. Time needed for operator to inform / to switch to the responsible for this type of emergency the correspondent service (e.g. police, ambulance or firefighters), then 6. Time for the service to find and to allocate the most suitable and / or the closest team, cars and / or a patrols to the place of emergency, give them the location and the explanations of what, when and where the emergency has happened. In case if it's a crime - to give signs and portraits of any possible suspects and their directions of the expected escape to organise a manhunt. 7. Additional time takes to give CCTV operators and / or an air crew or drone operators the location of the emergency / crime scene and explanation of who is suspected, what, with whom and when the crime happened. The explained above reaction time can be drastically shorten and all reactions be automated. The first responders activity before, during and after an emergency can be guided and assisted with the proposed system. 8. BRIEF SUMMARY OF THE INVENTION The proposed in the invention system works in the following manner: i) in an emergency one can press smartphone’s emergency button (e.g. 5 times quick press on “power” button or one can use any specially designed hardware or software) and this initiates ii) the preinstalled and set as a default Internet messenger, which starts record audio, photo and video at the scene of emergency immediately after the activation. The smartphone (if there is no a cancelation from the owner) iii) transmits this audio / photo / video with the location and the ID information of the owner and the phone through Internet to the address of iv) Al powered self-learning system which can recognise the type of emergency and v) can transmit the processed data (audio / photo / video) from the scene to the most suitable team of first responders with the vi) proposed optimal route to reach the scene and the possible recommended algorithm of actions on the scene. In addition the system vii) registering all communications connected with the emergency and can report them farther according to the existing protocols and instructions. Additionally the system, depending on the type of emergency, can give the victim(s) audio and video instructions about the most appropriate behaviour at the scene of emergency and communicates based on the answer of the victim(s)or the witnesses involved in an emergency. After the emergency resolved the system add adio / video records of it into the database for further use for an emergency detection. BRIEF DESCRIPTION OF THE DRAWINGS 1. Figure 1. Figure 1 shows the general architecture of the proposed system and the principal interactions of the main components. 2. Figure 2. Figure 2 illustrates the previous art where one can transmit only voice to an emergency operator and the proposed possible components which can be connected to and guided by the Al powered self-learning system for emergencies recognition, reaction and crime prevention. 3. Figure 3. Figure 3 explains what the components of the system already exist, should have or to be added with , their features and the interactions before an emergency occurred. 4. Figure 4. Figure 4 shows the interactions of the components of the system and the reaction steps during a knife attack. DETAILED DESCRIPTION AND BEST MODE OF IMPLEMENTATION The proposed first responders Al powered self-learning system for emergencies recognition, reaction and crime prevention (the system) (see Fig.1) consists of the: smartphone (101), equipped with a means for initiating (including in a clandestine mode) an emergency calls / message with the preinstalled on this smartphone any commercially available internet messenger or messengers (102) like Skype, Whatsllp, Telegram, Signal, Instagram, TikTok, FaceTime, etc. set in advance as a default messenger in an emergency and capable to capture, store and transmit audio and video data collected by the smartphone’s microphone and cameras so that the smartphone can transmit the data from the emergency scene live using any mobile operators (103) and / or other internet service provider’s networks to the paired in advance Internet messenger installed at the proposed Al powered machine-learning system (104), which is in communication with and / or located at the first responders operational room (106) and this system is capable to receive, to analyse and to transform data from the victim’s and / or witnesses’ smartphones. For the emergency recognition the system uses the specialised software (this is the Al part of the system) for the comparison the images from the scene of emergency with the preloaded and constantly updated database (105) consisted of the recordings of different emergencies. The system been able to self-learn from the previous experience of dealing with various emergencies by constantly adding more and more details and variants of the recorded b y the system emergencies to the database. The proposed system through “999” operators means of communications (107) is in connection with the first responders teams / crews, public safety CCTV operators, other means of the control and secure for the public spaces (108) so that the system, after receiving the emergency message from a smartphone can detect the type of emergency, the amount and the urgency of the help needed at the scene and allocates the most suitable first responders team(s) to deal with an emergency and supply the allocated responders with the processed and compatible with the existing first responders equipment information from the emergency site. System transmits the processed audio and video data about the optimal route to the scene of the emergency, nature of the emergency, who and what type of help the person(s) needs with the possible variants of actions at the place of emergency, simultaneously the system gives CCTV’s operators targeting of the place of emergency to get more data about an emergency and it’s actors. To minimise the reaction time, to exclude human bias and mistakes and to increase the efficiency of the actions during emergency situations the invention proposes to automate the following activity after the Initiation of the emergency cali: 1) Connection with the system and the transmission of the audio / video from the scene by the Internet to the system; 2) Recognition the type of emergency and an allocation the most suitable team(s) / crew(s) of first responders to deal with it; 3) Processing / transforming an emergency data into compatible with the existing first responders’ equipment (for example police officers’ radios, phones and cars communicators) format. This processed data can be visual and / or audio information about what, when, where, with whom and what type of emergency has happened. 4) Proposition to the allocated responders crew(s) / team(s) the optimal route and the order of possible actions at the scene with a fixation and report the results of the emergency actions according to the existing laws and instructions. The invention proposes the following additions and abilities to the existing first responders system: A) Internet service providers networks (fix,Wi-Fi, mobile, hot-spots etc.) with the B)The possibility to be used by any public smartphones to initiate and to transmit emergency calls / data and to connect by Internet to the system Internet messengers which can be of different types. Emergency calls / connection can be free and without the necessity to be registered on network or to have a positive balance on one’s account to be connected to Internet in such a manner to C) Automatically transmit the emergency calls / data / audio / video with the owner’s and the smartphone’s ID information also to “999” centres / operational rooms by Internet protocol employing any of the D) existing, preinstalled on the smartphones and set for an emergency communication Internet messenger(s) to transmit in real time (live) : ID data, location, audio, photo and video data from the scene of emergency (from victim’s and / or witnesses’ smartphone) to the processing software of the E) Artificial intelligence powered self-learning automated system for an emergency recognition, reaction and crime prevention connected or installed at the first responders HQs (“999” operational rooms) for almost instant detection of what, where, when and with whom the exact type of theemergency has happened so to F) Al component of the system, after recognition and classifying the emergency, can give directions, supply the information from the scene of the emergency to the chosen and the allocated by the system the most suitable correspondent team(s) or crew(s) of the first responders (e.g. reroute data to the allocated police crew / cars and to the officer’s smartphone the processed compatible video / photo / audio data from the scene of crime) , to CCTV operators, manned and unmanned (drones) aerial vehicles, patrols on the streets and others responders or services. The preferred mode of implementation of the proposed system can be the following : 1. Before an emergency (see Fig.3) At the “999” operational room / HQ / centres must be installed or connected to, tested, “trained” and commissioned the Al powered self-learning system for emergencies (300). This system consists of hardware (301.1) - computer servers with the installed software (300.2) which should recognise audio recordings and transform them into machine-readable format (voice-to-text) and also can be able to compare photo and video images with the preinstalled and constantly updated database of audio, images and video recordings (300.3) from the emergencies scenes collected from the different parts of a district / town / city / county / country. This database serves as a reference point and a learning tool for the system which can recognise / detect the type of emergency by comparing the images received from the victim’s and or witnesses’ smartphones from the scene of emergency with the referral images from the database. Based on this comparison the system detects the exact type of emergency, place, those who need help, possible injuries and what type of the help the victim(s) may need. It is necessary to add that additionally the system has the ability to control the location of the crews of the first responders in real time and has the preinstalled instructions for the first responders team tailored to different types of emergencies and can propose the optimal routes for the crews to reach the place of emergency and the expected and tailored to the scene algorithm of the actions on the site. The system also must be connected to existing means of communications (303) of first responders: Internet (303.1) and radio (Airwave) 303.2 and the system can sent them in the compatible with the emergency services communication equipment format processed and analysed audio / photo and video data added by the maps, routes and instructions and / or warnings of the risks one can expect on the scene. This material the system collects from the victim’s and / or witnesses’ of an emergency smartphones and transmits after analysis and transformation to the first responders. On any smartphone (302) which can be used for the interaction with the proposed system must be preinstalled two main components: Means to initiate an emergency call / message using hardware (302.1) (“power”, “volume” or any other physical knobs or attached device which, after pressing the programmed number of times or making any predefined manipulations (including clandestinely) connects to the system, switches on the initiated smartphone’s mic and cameras, starts recording and transmits the data to the system). In addition or instead of a hardware switch a software emergency call / message “trigger” (302.2) (preinstalled different applications which can, for example, be switched on by the code word or by any specified touchscreen actions - say knocking, scratching on the touchscreen in a special manner etc.) Besides inside an emergency call / message the smartphone can transmit its precise location and unique ID number / information (302.3) (besides phone’s data, like IMEI / serial number / model also can be added the personal data of the owner, medical issues, NINO, addresses, any relatives, contact numbers etc. This information must be set in advance.) Preinstalled one or several Internet messengers (302.4) (e.g. “Whatsllp”, “Facebook Messenger”, “FaceTime’, “Viber”, “Telegram”, “WeChat”, “Instagram”, “Signal” etc.etc.) which must be set in advance as a default emergency communicator(s) and be connected / paired with the Internet messenger installed on the system and this messenger should be able, after triggering ( by the mean 302.1 and / or 302.2) to record audio using the smartphone’s mic and photo and video using the smartphone’s cameras (alternately front and main cameras or in any time of the engagement of cameras the ratio) and to transmit it to the paired in advance an emergency Internet messenger of the system (300.5) to supply the Al self-learning system with the data from the scene of emergency in real time. Besides receiving data directly from the system the emergency services (305) also has communications with “999” operational rooms (304) and teams / crews / cars (from 305.1 till 305.7) “on the ground”. These teams / cars / patrols (313) can transmit own location to the system and to “999” operational room (304). 2. During an emergency For the understanding of the functioning of the system in a real life Fig.4 shows the work of the system in case of a knife crime / attack. Step 1: A victim or a witness of the attack triggers an emergency call / message by activating the smartphone (400) which was preliminary set to interact with the system as was explained above and illustrated by Fig.3. From the moment of the activation of the call the system can start to work fully autonomously and without any human interference in the following manner: Step 2: smartphone’s software switches on the predefined Internet messenger (401) which starts to record audio / video with backup it on the device and / or on any connected cloud storage; - the initiated smartphone (if there is no the “cancel an emergency call” actions from the owner during the “cancel because of mistake” period of time) connects through an Internet provider’s networks with the system’s Internet messenger (402) and / or “999” operational room (406) and starts to transmit to the system phone’s and owner’s ID, location, audio and video from the scene of emergency; - the system receives device’s ID information , location, audio / video data , compares this data with the database of emergencies, recognises the type of emergency - a knife attack, the help needed - first of all police and ambulance - allocates the case to the closest suitable teams and Step 3: supply police crew / patrols (403), CCTV operators (404), ambulance (405) and other responders with the data from the scene, the best routes to the place of attack, proposed order of the actions, data of the suspect(s) and their possible escape routes and the probable whereabouts. Step 4: The system controlled by the “999” operational room (406) staff. The system also informs / interacts with the staff and gives the data about the situation (collected from the smartphone, targeted CCTV cameras around the scene, from the first responders recording equipment and other sources. After the emergency resolved the system adds the recorded audio / video (407) from the scene to the database of emergencies (408) for a further use for an emergency recognition and reaction.

Claims

What is claim is:I.The first responders Al powered self-learning system for emergencies recognition, reaction and crimes prevention (the system) consists of:Smartphone (1.1.) equipped with a means for initiating an emergency calls with the preinstalled on this smartphone any commercially available Internet messenger (1.2.) set in advance as a default messenger to be used by the smartphone in a case of emergency and connected / paired in advance 1with the correspondent Internet messenger of the system. Mentioned smartphone is capable to capture, to store and to transmit audio and video data (1.3) collected by the smartphone’s microphone and cameras so that the smartphone can transmit the recorded after it’s initiation data from the emergency scene live to the system using any mobile operators and / or other Internet service provider’s networks (1.4.) to connect with the system’s Internet messenger and transmit and receive data to and from the system , which in the communication with and / or located at the first responders operational rooms / centres (e.g. “999”, “911”, “112” services) and this system is capable to receive, to analyse and to transform into other formats data from the victim’s and / or witnesses’ smartphones. For the emergency recognition the system uses (1.5.) specialised software which compares the audio recordings and images / video recordings from the scene of emergency with the preloaded and constantly updated audio / images / video from the database (1.6.) with the recordings of different emergencies in different conditions and places. The system is able to self-learn from the previous experience of dealing with the various emergencies by constantly adding more and more details and variants of the recorded by itself emergencies. The proposedsystem through the emergency services means of communications (1.7.) is in connection with the first responders teams / crews, public safety CCTV operators, other means of the control and secure for the public spaces (1.8.) so that the system, after receiving the emergency message and audio / video from a scene of emergency can detect the type of emergency, the amount and the urgency of the help needed at the scene and allocates the most suitable first responders team(s) (1.9.) to deal with an emergency and supply the allocated responders with the processed and compatible with the existing first responders equipment information from the emergency site. System transmits the processed audio and video data about the optimal route to the scene of the emergency (1.10.), nature of the emergency, who and what type of help the persons involved need (1.11) with the possible variants of actions at the place of emergency (1.12), possible risks at the scene (1.13) and other useful for the responders information.

2. The system according to the claim 1 works in the following manner:In an emergency one can press smartphone’s emergency button (2.1.) which can be part of the smartphones hardware or software and this initiates the preinstalled and set as a default Internet messenger (2.2.), which starts record audio / photo / video (2.3.) at the scene of emergency immediately after the activation. The smartphone transmits this audio / photo / video with the location and the owner’s and the phone’s ID information (2.4.) through Internet service providers networks to the Internet messenger of the system (2.5.). The system analysed the received data and can recognise the type of emergency (2.6.) and can transmit the processed data (audio / photo / video) from the scene to the most suitable team(s) of first responders (2.7.) with the proposed optimal route to reach the scene and the possible recommendedalgorithm of actions (2.8.) on the scene. In addition the system registers all communications connected with the emergency (2.9.) and can report them farther according to the existing protocols and instructions. Additionally the system, depending on the type of emergency, can give to the victim(s) audio and video instructions (2.10) about the most appropriate behaviour at the scene of emergency and communicates bases on the answer of the victim(s)or the witnesses involved in an emergency. After the resolving of the emergency the system add the audio / video recordings of this case (2.11.) into the database for the use for a next time an emergency detection and reaction.3.The mentioned in the claim 1 smartphone must have a microphone and cameras (main camera(s) and a “selfy” one) to record audio and video around this smartphone after triggering.

4. The mentioned in the claim 1 Internet messenger can be one or several from the publicly available messengers for example “Whatsllp”, “Facebook Messenger”, “FaceTime’, “Viber”, “Telegram”, “WeChat”, “Instagram”, “Signal” and alike which can record and transmit audio, video and phone’s ID to the system where the different types of Internet messengers also installed. The chosen Internet messenger must be installed on the phone, set and be connected / paired to the system’s messenger (added to the list of the “allowed” contacts) in advance before the emergency.

5. Means to initiate an emergency call / message by smartphone mentioned in Claim 1 can be part of this phone’s hardware (“power”, “volume” or any other physical knobs or attached device which, after pressing the programmed number of times or making any predefined manipulations (including clandestinely) connects to the system, switches on on the initiated smartphone’s mic and cameras, starts recording and transmits the data to the system. Inaddition or instead of a hardware switch a software emergency call / message “trigger” can be used. Such as preinstalled different applications which can, for example, be activated by the code word or by any specified touchscreen actions - say knocking, scratching on the touchscreen in a special manner to form a special predefined patterns.

6. The system according to claim 1 has the possibility to communicate with the victim(s) of emergency through Internet messenger and to advice the possible actions on the scene of emergency and / or to switch on a siren and / or a flashlight so to ease the discover of persons who maybe injured or inside buildings or can not respond or move or unconscious. Emergency operators and first responders also can use said Internet messenger to communicate with the victim before arriving to the scene.

7. The system according to the claim 1 can be used not only for emergencies management but also by other services and officials to collect actual outdoor and indoor information for different purposes from fixation the Highway Code violations, control over persons of interest till discover the persons at large.

Images