Notification generation method, mobile processing device and system for resource-constrained device

Abstract

The present invention relates to a computer-implemented method for offloading notification processing comprising the steps of: receiving an electronic communication on a mobile processing device; generating a notification on the mobile processing device based on the electronic communication; generating a notification image on the mobile processing device based on the notification, by converting a part of the notification comprising at least one character into an image; transmitting the notification image from the mobile processing device to a resource-constrained device through a wireless network. It also relates to a mobile processing device and a system for offloading notification processing adapted to carry out the method above.

Description

Notification generation method, mobile processing device and system for resource-constrained deviceTechnical domain

[0001] The present invention concerns a computer-implemented method, a mobile processing device and a system for generating notifications for resource-constrained devices.Related art

[0002] The rise of resource-constrained devices has recently been driven by a number of factors, including the growth of the loT, the expansion of edge computing, cost considerations, a focus on energy efficiency, advances in miniaturisation, and the ubiquity of connectivity technologies. These devices are designed for specific tasks with limited resources to balance functionality, efficiency and cost.

[0003] Due to their resource-constrained nature, these devices typically have low processing capacities caused by hardware limitations and / or limited autonomy in terms of power supply. This shortage in processing abilities severely contrasts with the increasing number of functionalities provided by modern telecommunication systems which can be demanding in terms of computing power and / or power use.

[0004] The hardware limitations of resource-constrained devices may, for example, result in low on-board memory capacity that prevents the device from storing data that can be used to display or process communications to the user.

[0005] Typical electronic messages typically contain characters. In this context, the term "character" indicates an abstract entity representing asymbol or a unit of writing, such as a letter, a number, a punctuation mark, or an emoji. "A", "7",B " are non-limiting examples of characters.

[0006] In general, the characters are not transmitted as such, but they are encoded (in general by a coding system as ASCII, Unicode, etc. e.g. for a letter, or for an emoji, or by association to a unique string e.g. for an emoji, though this way of encoding has been replaced by the Unicode standard in recent years) by the transmitting mobile processing device, so that the encoded character is transmitted to a receiving mobile processing device. In the receiving mobile processing device, the encoded character is decoded, so that the corresponding character is then displayed. The decoding and displaying process requires memory and / or computing power. Hence, both the transmitting and the receiving device must have dedicated storage and processing capacity.

[0007] Moreover, some alphabets require special text processing within the terminal to display the original text correctly. For example, Arabic and Hebrew include letters, accents, etc. that are position dependent, e.g. they change according to their location within a word or a sentence (contextual shaping). As a result, punctuation may need to be specifically aligned according to the position of those letters, and / or the connection between successive letters may need to be computed to achieve a visually satisfying rendering. All those specificities make text processing computationally intensive, but also requires more memory to store all the glyphs required as well as the code executed for performing the rendering of the original text on a display.

[0008] As a consequence, notifications including such characters need more space in the memory storage of the transmitting devices and on the receiving devices, in particular for the representation or rendering of the character.

[0009] Although resource-constrained devices may not be designed to display the full content of an electronic message such as an SMS, email orinstant message, they typically support at least a notification service that alerts the user of the resource-constrained device that an electronic communication has been received.

[0010] Such notifications may include various forms of visual information, such as structured / unstructured text, a fraction of the content of the communication, a logo, an image, etc. The processing cost of generating and / or displaying such a notification may therefore be too high for this action to be performed on a resource-constrained device without compromising its autonomy and / or processing capabilities.

[0011] Document US20150020081 A1 discloses a method for notifying a user of an external device that a message has been received on a digital device paired with the external device. A notification, possibly integrating visual elements, is generated in the digital device and transmitted as a whole to the external device. In particular, this necessitates to transmit from the digital device to the external device, all information required to process the notification on the external device (e.g. font, line breaks, particular encoding), and the external device needs to have enough computing power to process the notification and / or enough on-board memory to store databases (e.g. characters encoding databases).

[0012] It is therefore desirable to develop solutions allowing to notify the user of a resource-constrained device that an event occurred on a paired device which require his attention, while taking into account the hardware and / or software limitations of the resource-constrained device.Short disclosure of the invention

[0013] An aim of the present invention is the provision of a method and a system for notification processing on a resource-constrained device that overcomes the shortcomings and limitations of the state of the art.

[0014] Another aim of the invention is to provide a method and a system allowing a resource-constrained device to display a wider range of notifications.

[0015] Another aim of the invention is to provide a method and a system increasing the autonomy of resource-constrained devices capable of displaying notifications.

[0016] According to the invention, these aims are attained by the object of the attached claims, and especially by a computer-implemented method for offloading notification processing comprising the steps of: receiving an electronic communication by a mobile processing device; generating a notification by the mobile processing device based on the electronic communication for notifying a user of the mobile processing device that the electronic communication has been received on the mobile processing device; generating a notification image by the mobile processing device based on the notification, by converting a part of the notification comprising at least one character into an image; transmitting the notification image from the mobile processing device to a resource-constrained device through a wireless network, so as to notify a user of the resource-constrained device that a notification has been received on the mobile processing device.

[0017] Advantageously, delegating all the processing related to the generation of the notification to the mobile processing device allows the resource-constrained device to be able to display characters that would normally need processing / rendering to by displayed. In particular, it becomes possible for resource-constrained devices to display emojis or characters of languages whose rendering need additional processing, e.g. Arabic or Hebrew.

[0018] Another advantage of the invention is that it allows a resource- constrained device to display characters that were not originally supportedby the firmware / software of the device without the need of updating them. In particular, it allows the resource-constrained device to display new emojis or new languages without updating the embedded firmware / software.

[0019] According to some embodiments, delegating all the processing related to the generation of the notification to the mobile processing device advantageously allows to preserve the battery of the resource- constrained device and extend its autonomy.

[0020] In this context, the expression "converting a part of the notification comprising at least one character into an image" indicates that a portion of the notification containing at least one character is transformed into an image. In other words, the notification image comprises an image representation of the at least one character.

[0021] Surprisingly, generating an image representation of at least one character of the notification, and possibly of the notification itself generated on the mobile processing device and transmitting this notification image instead of the notification itself allows to save power on the resource-constrained device by offloading the processing of the notification from the resource-constrained device.

[0022] In one embodiment, the computer implemented method comprises the step of displaying the notification image on a display of the resource-constrained device.

[0023] In one embodiment, the computer implemented method comprises, after the step of generating a notification and before the step of generating a notification image, a step of processing the notification by the mobile processing device.

[0024] In one embodiment, the step of processing the notification by the mobile processing device comprises text shaping, word and / or line breaking, bidirectional text support and / or text rendering. This step, which is useful in particular if both RTL (right to left) and LTR (left to right) languages are used, is performed in the mobile processing device, and not in the resource-constrained device. In fact, when a notification contains both language types, the text flow adjusts accordingly, with bidirectional control characters ensuring that text displays correctly, such as when combining an Arabic sentence with English words in the middle. The complexity of managing both directions of text is why bidirectional text support is important in software development and localization and in this embodiment the bidirectional text support is performed in the mobile processing device.

[0025] In one embodiment, the at least one character in the notification is a letter of a RTL language, e.g. an Arabic character and / or a Hebrew character.

[0026] In one embodiment, the method further comprises a step of compressing the notification image to reduce a transmission cost of the notification image from the mobile processing device to the resource- constrained device.

[0027] The resource constrained device can comprise a battery with a capacity of less than 1 Ah.

[0028] The notification generated by the mobile processing device can be a push notification for notifying a user of the reception of an electronic communication.

[0029] In an embodiment, the resource constrained device is a wearable device such as a connected watch, a connected bracelet, a fitness tracker, a connected health monitoring device or an augmented reality device.

[0030] In one embodiment, the mobile processing device is a smartphone, a tablet, a PC or a connected watch.

[0031] In an embodiment, the step of transmitting the notification image from the mobile processing device to a resource-constrained device is executed following a network protocol selected among: Wi-Fi, Bluetooth, Bluetooth Low Energy, AMQP, Cellular, CoAP, DDS, LoRa, LoRaWAN, XMPP, LWM2M, MQTT, Z-Wave, Zigbee, or any proprietary radio protocol. In particular, low power consumption protocols as Bluetooth Low Energy (BLE) and LoRaWAN are preferred.

[0032] In one embodiment, the format of the notification image is selected among the following formats: BMP, AVIF, PNG, SVG, JPG / JPEG, BPG, WebP, CLUT-based format or any proprietary image format.

[0033] These goals are also achieved by means of a mobile processing device characterized in that it is adapted to carry out the computer implemented method described above.

[0034] In one embodiment, the mobile processing device is a smartphone, a tablet, a PC or a connected watch.

[0035] These goals are also achieved by means of a system for offloading notification processing comprising: a mobile processing device according to the invention; a resource-constrained device connected to the mobile processing device through a wireless network.

[0036] In one embodiment, the resource constrained device is a wearable device such as an electronic watch, a connected bracelet or an augmented reality device.

[0037] In one embodiment, the wireless network operates following a network protocol selected among: Wi-Fi, Bluetooth, Bluetooth Low Energy,AMQP, Cellular, CoAP, DDS, LoRa, LoRaWAN, XMPP, LWM2M, MQTT, Z- Wave, Zigbee, or any proprietary radio protocol. In particular, low power consumption protocols as Bluetooth Low Energy (BLE) and LoRaWAN are preferred.Short description of the drawings

[0038] Exemplar embodiments of the invention are disclosed in the description and illustrated by the drawings in which:Fig. 1 is a flowchart of the main steps of the invention.Fig. 2 is a flowchart of the main steps of the invention with an optional compression step.Fig. 3 illustrates schematically a system comprising a mobile processing device wirelessly connected to a resource-constrained device.Examples of embodiments of the present invention

[0039] The present invention concerns a computer implemented method for offloading the processing of a notification from a resource-constrained device 30 having limited or restricted processing power.

[0040] In the context of the present disclosure, the expression "resource- constrained device" designates a device having limited or even no processing and storage capabilities designed for low energy consumption. Such devices usually have limited energy supply, typically limited battery life. As a limitation, the considered resource-constrained devices must include a display capable of displaying an image to its user.

[0041] The capacity, or autonomy, of a resource-constrained device is usually measured in ampere-hour (Ah). In one embodiment, the resource- constrained device of the present invention has a capacity of at most 1 Ah, preferably at most 500 mAh (milli ampere-hour).

[0042] In particular, resource-constrained devices as considered in this text include all kind of wearable devices such as connected watches (also called smartwatches), connected bracelets, connected health monitoring devices, fitness trackers, safety / location devices, smart glasses, augmented reality devices, smart clothing, hearable devices, payment wearables, intelligent patches (e.g. price tags), etc.

[0043] All those resource-constrained devices 30, an example of which is visible in Figure 3, are intended to be wirelessly connected to a mobile processing device 20, an example of which is also visible in Figure 3, having higher processing, memory and / or power supply capacities. Typically, this category of mobile processing devices includes smartphones, tablets, gaming devices, wearables with high process! ng / memory / battery capacity, laptops, PCs etc. More generally, any mobile device capable of efficiently (i.e. without compromising its basic functionalities) processing electronic communications is considered to be a mobile processing device.

[0044] Some of the functionalities of the mobile processing device 20 can be extended to the resource-constrained device 30. In particular, the resource-constrained devices are well-suited to alert its user that an event occurred on the mobile processing device, typically by means of a notification (also called "push notification"). Indeed, these connected resource-constrained devices are usually disposed in the direct proximity or even worn by their users so that providing a notification on those devices has an increased chance to get the user's attention.

[0045] However, as mentioned above, the generation (i.e. the processing and / or rendering) of a notification based on a received electronic communication can necessitate extensive processing from thedevice and / or extensive onboard memory. Such requirements are usually not met by resource-constrained devices. As a result, these requirements prevent those resource-constrained devices from displaying such notifications, especially when they contain elements requiring a specific rendering (emojis, specific languages such as Arabic, Hebrew, Cyrillic, etc.).

[0046] Moreover, in case new languages, new emojis or new characters are to be used to generate notifications, the databases on which the rendering relies to process the notifications need to be updated. This may involve database, software and / or firmware update on the device performing the rendering. Such frequent updates are not well-suited for the use of resource-constrained devices. In fact, resource-constrained devices may not have enough memory for new languages, new emojis or new characters to be used

[0047] The expression "electronic communication" refers to any type of digital message that can be sent to a mobile processing device 20 including, but not limited to, emails, instant messaging (i.e. real-time text-based communications), text messaging (i.e. SMS), voice over internet protocol (VoIP), video conferencing, phone calls, etc. The electronic communication may also refer to a digital message emitted by the mobile processing device 20 itself, for example to provide information on a process being executed on the mobile processing device 20 (e.g. software installation status, information about a battery status, alarm, etc).

[0048] Hence an electronic communication can refer to any digital communication between humans, between a processing device and a human or even between processing devices without any human intervention. Any such communication can give rise to a notification destinated to the user of the mobile processing device.

[0049] With reference to Fig. 1 the method comprises a first step of receiving an electronic communication 10 on a mobile processing device 20.

[0050] In reaction to the reception of an electronic communication, the method comprises a second step of generating a notification 11 on the mobile processing device 20 based on the electronic communication.

[0051] In a preferred embodiment, the mobile processing device 20 is a smartphone / tablet. In this typical embodiment, the notification can be generated by the operating system and / or directly by a dedicated software such as a mobile app, depending on the type of electronic communication.

[0052] In the present application, the expression "notification" refers to any processed element based on a received electronic communication and whose purpose is to indicate to a user that an electronic communication has been received. Such notifications can be processed based on elements transmitted via the electronic communication such as characters and based on a set of rules that can be stored on the receiving device or transmitted together with the communication.

[0053] A notification therefore typically comprises several different elements such as characters (e.g. letters, numbers, punctuation marks, emojis, etc.), a logo (e.g. the logo of a mobile app to which the electronic communication relates), an image (e.g. the picture associated to a contact from which the communication is coming, a frame of a video), etc. However, in the broadest case, a notification may comprise only one character. Moreover, it can be displayed in various ways such as in the form of a banner, a pop-up, etc.

[0054] Advantageously the notification can be a push notification, i.e. an alert such as a pop-up or other message generated by an application when the application has not yet been opened to consult the content of the related communication. A push notification typically serves to notify the user of a new message, update, social media post, etc.

[0055] Advantageously, the present invention proposes to delegate processing tasks on the notification to the mobile processing device towhich the resource-constrained device is connected. After the notification has been generated by the mobile processing device, a notification image comprising at least an image representation of a character of the notification is generated by the mobile processing device from the notification and transmitted to the resource-constrained device for displaying it on a display of the device. It thus preserves the resource- constrained device from processing the notification as described above and allows it to display notifications containing particular characters, emojis, languages, etc., to display new content (emojis, characters, etc.) without the need of updates of the software / firmware and allows to extend the functionality of the resource-constrained device.

[0056] The notification image can therefore be viewed as a text-to- image conversion, i.e. an image conversion of at least one character present in the notification. The notification image may include additional elements such as another image, comprised in the notification.

[0057] Therefore, instead of transmitting encoded characters to the resource-constrained device and performing a power-consuming processing to generate the notification based on the encoded characters on the resource-constrained device, or not being able to display the character as its representation is not available in the memory of the resource constrained device, the present invention advantageously proposes to transmit an image, typically a low-resolution or small image, representing the notification generated in the mobile processing device 20.

[0058] In particular, it is possible to easily display notifications images representing characters or string of characters that need complex decoding tasks from their encoded form. For example, Arabic language or Hebrew language that need right-to-left encoding and / or ligature encoding in addition to the characters themselves can be easily used to notify the user on the resource-constrained device without the need for on-board libraries to decode the encoded characters.

[0059] Hence, the third step of the present method comprises the generation of a notification image 12 on the mobile processing device 20 based on the notification.

[0060] Alternatively or complementarily, the generation of notification image is preceded by one or more processing steps. As a non-limitative example, such a conversion may involve generating from the textual content of the notification a framebuffer displaying the text and converting into an image.

[0061] In an embodiment, only a fraction of the notification or of its content is considered to generate the notification image. This can serve the purpose of reducing the size of the notification image and / or can be purely for visual considerations.

[0062] It may also be required to reduce the size of the notification to create a notification image that can be efficiently transmitted to the resource-constrained device. Therefore, the method may include an optional image compression step 15 as illustrated in Fig. 2. This compression step may be part of the generation of the notification image as most of image formats suitable for efficient transmission already include lossless or lossy compression in the encoding. However, it may also be an additional compression step in order to comply with the wireless network and protocol used for transferring data from the mobile processing device 20 to the resource-constrained device 30.

[0063] In an embodiment, this step is carried out by a dedicated software in the mobile processing device 20. Its execution causes the generated notification to be retrieved and a notification image to be created based on the notification.

[0064] Habitually, the steps of generating the notification itself are carried out by the existing OS via and / or by dedicated software (such as anapp) of the mobile processing device connected to the resource-constrained device 30.

[0065] Although the notification image can be virtually in any image format, it is advantageous in the context of the present invention that it is an image format suited to energy efficient transmission. Typically Colour- Lookup Table (GLUT) based format or compressed image formats are well suited for the notification image. In particular, the notification image can be generated in a format such as AV1 Image File Format (AVIF), Portable Network Graphic (PNG), Scalable Vector Graphic (SVG), Joint Photographic Expert Group (JPG / JPEG), Better Portable Graphics (BPG), WebP, and / or any custom proprietary format.

[0066] The format of the notification image 13 is typically guided by the constraints that it has to be easily handled (e.g. simple bitmap) and / or compressed (e.g. CLUT-based).

[0067] The method comprises a fourth step of transmitting the notification image 13 from the mobile processing device 20 to a resource- constrained device 30 through a wireless network 40.

[0068] Numerous wireless connection protocols can be implemented on the wireless network. As per the image format, energy efficient (lowconsuming) protocols are privileged in the context of the present invention.

[0069] In an embodiment, the wireless protocol is selected among the following protocols: Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), Advance Message Queuing Protocol (AMQP), Mobile / Cellular protocols, Constrained Application Protocol (CoAP), Data Distribution Service (DDS), Long-Range Wide-Area Network (LoRa / LoRaWAN), Extensible Messaging and Presence Protocol (XMPP), Lightweight Machine to Machine (LWM2M), Message Queuing Telemetry Transport (MQTT), Z-Wave or Zigbee. More generally, any proprietary radio protocol can be used for transmitting the notificationimage from the mobile processing device 20 to the resource-constrained device 30.

[0070] After its transmission to the resource-constrained device 30, the notification image can be displayed 14 on a display 31 of the resource- constrained device 30. The display 31 can be any type of display capable of displaying an image file, typically LCD, OLED, AMOLED, TFT, etc. Usually resource-constrained devices privilege low-consuming display technology to extend the battery life. The display 31 is therefore preferentially a low power consuming display.

[0071] In a preferred embodiment illustrated in Fig. 3, the mobile processing device 20 is a smartphone and the resource-constrained device 30 is a smartwatch connected to the smartphone via a wireless network 40, preferably via Wi-fi or BLE. The method of the present application can therefore be executed on those devices by:• receiving an electronic communication on the smartphone, typically an email or an instant message;• generating a notification on the smartphone using a dedicated app or the mobile OS of the smartphone to notify a user that an electronic communication has been received (from an external device or from the smartphone itself);• generating a notification image based on the notification by converting a part of the notification comprising at least one character into an image, typically in an image format adapted to narrow band data transmission such as WebP, SVG, BMP or CLUT- based format;• transmitting the notification image from the smartphone / tablet to the smartwatch using a wireless network such as Wi-Fi, BLE, etc, to notify its wearer of the received electronic communication.

[0072] In another embodiment, the resource-constrained device is a low- power intelligent patch, which is able to render pre-computed content, but have no processing capabilities, neither enough memory to stores glyph, such patch can be seen as a battery powered intelligent price tags. In this case the mobile processing unit could broadcast the notification image to the battery powered intelligent price tag.

[0073] The present invention is also related to a mobile processing device 20 adapted for carrying out the steps of the method described above.

[0074] The present invention is also related to a system adapted for carrying out the steps of the method described above and comprising a mobile processing device 20 and a resource-constrained device 30 connected to the mobile processing device through a wireless network 31.

[0075] In particular, the mobile processing device 20 is adapted for carrying out the steps of receiving an electronic communication; generating a notification based on the electronic communication for notifying a user that an electronic communication has been received on the mobile processing device; generating a notification image based on the notification, by converting a part of the notification comprising at least one character into an image, transmitting the notification image from the mobile processing device to a resource-constrained device through a wireless network, so as to notify a user of the resource-constrained device that a notification has been received on the mobile processing device.

[0076] The resource-constrained device 30 is adapted to display the notification image on a display.

[0077] Throughout the present disclosure, the user of the mobile processing device 20 and the user of the resource-constrained device 30 may be the same or may be two different users.

[0078] The mobile processing device of the system is typically a smartphone, a tablet, a PC or a connected watch.Reference numerals Reception of electronic communication Notification generation Notification image generation Wireless transmission of the notification image Display of the notification image on resource-constrained device Notification image compression Mobile processing device Resource-constrained device Display of the resource-constrained device Wireless network

Claims

Claims1. Computer-implemented method for offloading notification processing comprising the steps of: receiving an electronic communication (10) by a mobile processing device (20); generating a notification (11) by the mobile processing device (20) based on the electronic communication for notifying a user of the mobile processing device that the electronic communication has been received on the mobile processing device; generating a notification image (12) by the mobile processing device (20), by converting a part of the notification comprising at least one character into an image; transmitting the notification image (13) from the mobile processing device (20) to a resource-constrained device (30) through a wireless network (40), so as to notify a user of the resource-constrained device that a notification has been received on the mobile processing device (20).

2. Computer implemented method according to claim one, comprising the step of displaying the notification image (14) on a display (31) of the resource-constrained device (30).

3. Computer implemented method according to any of the claims 1 to 2, comprising, after the step of generating a notification and before the step of generating a notification image, a step of processing the notification by the mobile processing device.

4. Computer implemented method according to claim 3, wherein the processing of the notification by the mobile processing device comprises text shaping, word and / or line breaking, bidirectional text support and / or text rendering.

5. Computer implemented method according to any of the claims 1 to 4, wherein the character is a letter, a number, a punctuation mark and / or an emoji.

6. Computer implemented method according to any of the claims 1 to 5, wherein the character is a letter of a RTL language, e.g. an Arabic character and / or a Hebrew character.

7. Computer implemented method according to any of the claims 1 to 6, further comprising a step of: compressing the notification image (15) to reduce a transmission cost of the notification image from the mobile processing device (20) to the resource-constrained device (30).

8. Computer implemented method according to any of the claims 1 to 7, wherein the resource constrained device (30) is a wearable device such as a connected watch, a connected bracelet, a fitness tracker, a connected health monitoring device, an augmented reality device or an intelligent patch.

9. Computer implemented method according to any of the claims 1 to 8, wherein the resource constrained device (30) comprises a battery with a capacity of less than 1 Ah.

10. Computer implemented method according to any of the claims 1 to 9, wherein the mobile processing device (20) is a smartphone, a tablet, a PC or a connected watch.

11. Computer implemented method according to any of the claims 1 to 10, wherein the step of transmitting the notification image (13) from the mobile processing device (20) to a resource-constrained device (30) is executed following a network protocol selected among: Wi-Fi, Bluetooth,Bluetooth Low Energy, AMQP, Cellular, CoAP, DDS, LoRa, LoRaWAN, XMPP, LWM2M, MQTT, Z-Wave or Zig bee.

12. Computer implemented method according to any of the claims 1 to 11, wherein a format of the notification image is selected among: AVIF, PNG, SVG, JPG / JPEG, BPG ,WebP, BMP or CLUT-based format.

13. A mobile processing device (20) for offloading notification processing characterized in that it is adapted to carry out the computer implemented method of any of the claims 1 to 12.

14. The mobile processing device (20) according to claim 13, being a smartphone, a tablet, a PC or a connected watch.

15. System for offloading notification processing comprising the mobile processing device (20) of one of claims 13 or 14; a resource-constrained device (30) connected to the mobile processing device through a wireless network (40).

Images