File processing method and device, storage medium and program product

Abstract

The invention provides a file processing method and device, a storage medium and a program product, relates to the technical field of image processing, and is beneficial to improving the image processing efficiency. The method comprises the steps that a to-be-decoded file is acquired, the to-be-decoded file comprises an image and associated information of the image, and the associated information of the image is used for describing the image or feature attributes in a target area of the image; analyzing the to-be-decoded file to obtain an image and associated information of the image; and performing corresponding operation on the image based on the associated information of the image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to a file processing method and device, a storage medium and a program product. BACKGROUND

[0002] In the technical field of computer vision, when an image is processed, such as image recognition, some image-related information is usually generated, such as recognized text information, object information, scene information and the like contained in the image. If the above image-related information is not saved in the picture or video file format, it will be lost in the transmission and reprocessing process, resulting in that the recognized information cannot be used across platforms or the same information is repeatedly generated, and computing resources are wasted. For example, the text in a picture can be recognized on an Apple mobile phone, but after the picture is transmitted to an Android mobile phone, the recognized text cannot be directly used, unless computing resources are used again to perform text recognition. SUMMARY

[0003] The embodiments of the present disclosure provide a file processing method, device, storage medium and program product, which are beneficial to improving the image processing efficiency. The technical solutions provided by the embodiments of the present disclosure are as follows.

[0004] In one aspect, a file processing method is provided, which includes:

[0005] obtaining a to-be-decoded file, the to-be-decoded file including an image and image-related information, the image-related information being used to describe a feature attribute in the image or a target region of the image;

[0006] parsing the to-be-decoded file to obtain the image and the image-related information;

[0007] performing a corresponding operation on the image based on the image-related information.

[0008] In another aspect, a file processing method is provided, which includes:

[0009] obtaining an image and image-related information, the image-related information being used to describe a feature attribute in the image or a target region of the image;

[0010] encapsulating the image-related information into the image to obtain a to-be-decoded file;

[0011] sending the to-be-decoded file.

[0012] In yet another aspect, a file processing device is provided, which includes:

[0013] an obtaining module, configured to obtain a to-be-decoded file, the to-be-decoded file including an image and image-related information, the image-related information being used to describe a feature attribute in the image or a target region of the image;

[0014] a parsing module, configured to parse the to-be-decoded file to obtain the image and the associated information of the image;

[0015] a processing module, configured to perform a corresponding operation on the image based on the associated information of the image.

[0016] In another aspect, a file processing apparatus is provided, which comprises:

[0017] an obtaining module, configured to obtain the image and the associated information of the image, the associated information of the image being used to describe a feature attribute in the image or a target region of the image;

[0018] an encapsulating module, configured to encapsulate the associated information of the image into the image to obtain the to-be-decoded file;

[0019] a communication module, configured to send the to-be-decoded file.

[0020] In another aspect, a communication apparatus is provided, which comprises a memory and a processor; the memory and the processor are coupled; the memory is configured to store computer program instructions executable by the processor; and the processor implements the file processing method of any of the above embodiments when executing the computer program instructions.

[0021] In another aspect, a computer readable storage medium is provided, which stores computer program instructions; and the computer program instructions implement the file processing method of any of the above embodiments when executed on a computer (for example, the communication apparatus or the file processing apparatus).

[0022] In another aspect, a computer program product is provided, which comprises computer program instructions; and the computer program instructions implement the file processing method of any of the above embodiments when executed.

[0023] The technical solution provided by the embodiments of the present disclosure comprises the following steps: obtaining a to-be-decoded file, the to-be-decoded file comprising an image and associated information of the image, the associated information of the image being used to describe a feature attribute in the image or a target region of the image; parsing the to-be-decoded file to obtain the image and the associated information of the image; and performing a corresponding operation on the image based on the associated information of the image. In this way, the associated information of the image and the image are jointly stored, so that the associated information of the image can be seamlessly transmitted and shared between different platforms and devices, so that different platforms and devices can perform corresponding operations on the image by parsing the associated information, thereby improving the image processing efficiency. At the same time, the process of repeatedly identifying the image to obtain the associated information of the image can be avoided, which is conducive to realizing the compatibility and portability of information. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1A schematic diagram illustrating an optional application scenario of a file processing method provided in this disclosure embodiment;

[0025] Figure 2 A flowchart of a file processing method provided in this embodiment of the disclosure;

[0026] Figure 3 A flowchart illustrating another file processing method provided in this disclosure embodiment;

[0027] Figure 4 This is a schematic diagram of the structure of a document processing device provided in an embodiment of the present disclosure;

[0028] Figure 5 This is a schematic diagram of another document processing apparatus provided in an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of a communication device provided in an embodiment of this disclosure. Detailed Implementation

[0030] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0031] In this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0032] It should be noted that in this disclosure, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0033] In the field of computer vision, image processing, such as image recognition, typically generates associated information, such as text, object, and scene information contained within the recognized image. If this associated information is not stored in the image or video file format (e.g., stored within the application itself rather than the image file), it will be lost during transmission and reprocessing. This results in the recognition information being unusable across platforms or the same information being generated repeatedly, wasting computational resources. For example, text can be recognized in an image on an iPhone, but once the image is transferred to an Android phone, the recognized text cannot be used directly without performing another text recognition operation.

[0034] With the continuous development of image generation and processing technologies, the application of artificial intelligence (AI) in image processing is becoming increasingly widespread, encompassing not only applications for ordinary users but also image or video processing tasks for enterprises. During image or video processing, related information is typically generated, such as text, target information, and scene information contained within the image. Currently, this information is usually generated and used only within the application and cannot be used across platforms.

[0035] In related technologies, text associated with images mainly falls into two categories:

[0036] 1. User description: Used to provide user description / annotation information that is readable by the human eye.

[0037] definition:

[0038] Box type: 'udes';

[0039] Attribute type: Descriptive item property;

[0040] Container: ItemPropertyContainerBox;

[0041] Required (for each associated item_ID): No;

[0042] Quantity (per associated item_ID): 0 or more.

[0043] UserDescriptionProperty allows you to associate items or groups of entities with user-defined names, descriptions, and tags; there may be multiple such properties, each with different language codes.

[0044] When several instances of a user-descriptive attribute are associated with the same item or entity group, they represent alternatives that may be expressed in different languages, and the reader should choose the most appropriate one. At most one user-descriptive attribute with the same `alt_lang` value can be applied to the same item or entity group.

[0045] grammar:

[0046] aligned(8)class UserDescriptionPropertyextends ItemFullProperty('udes',version=0,flags=0){

[0047] utf8string lang;

[0048] utf8 string name;

[0049] utf8 string description;

[0050] utf8string tags;

[0051] }

[0052] Semantics:

[0053] `lang` is a string containing language tag strings that conform to relevant standards, such as "en-US", "fr-FR", or "zh-CN", indicating the language of the text contained in the name, description, and tags. When `lang` is empty, the language is unknown / undefined.

[0054] `name` is a null-terminated UTF-8 string containing the human-readable name of the project or entity group. If it does not exist (an empty string is provided), no name is provided.

[0055] The description is a null-terminated UTF-8 string containing a human-readable description of the item or group of entities. If it does not exist (an empty string is provided), no description is provided.

[0056] `tags` is a null-terminated UTF-8 string containing comma-separated user-defined tags related to the project. If none exists (an empty string is provided), no tags are provided.

[0057] 2. Accessibility text: Used to display corresponding text information when an image cannot be displayed.

[0058] definition:

[0059] Box type: 'altt';

[0060] Attribute type: Descriptive item attribute;

[0061] Container: ItemPropertyContainerBox;

[0062] Mandatory (per item): None;

[0063] Quantity (per item): 0 or more.

[0064] The AccessibilityTextProperty contains a string that can be used as alternative text for an image if it cannot be displayed, similar to all text in HTML. The language used for the alternative text is represented by a language markup string that conforms to the relevant standards.

[0065] When several instances of an accessibility text attribute are associated with the same item, they represent options that may be expressed in different languages, and the reader should choose the most appropriate one. At most one accessibility text attribute with the same alt_lang value can be applied to the same item.

[0066] grammar:

[0067] aligned(8)class AccessibilityTextPropertyextends ItemFullProperty('altt',version=0,flags=0){

[0068] utf8string alt_text;

[0069] utf8string alt_lang;

[0070] }

[0071] Semantics:

[0072] alt_text is a string that can be used as alternative text for an image when it cannot be displayed, similar to the alt text in HTML.

[0073] alt_lang is a string containing language tag strings conforming to relevant standards, such as "en-US", "fr-FR", or "zh-CN", indicating the language of the text contained in alt_text. When alt_lang is empty, the language is unknown / undefined.

[0074] In view of this, this disclosure provides a file processing method. The method involves acquiring a file to be decoded, which includes an image and associated image information. The associated image information describes the feature attributes of the image or a target region within the image. The file to be decoded is then parsed to obtain the image and its associated information. Based on the associated image information, corresponding operations are performed on the image. This method combines the associated image information with the image itself, enabling seamless transmission and sharing of the associated image information across different platforms and devices. This allows different platforms and devices to perform corresponding operations on the image by parsing this associated information, thereby improving image processing efficiency. Furthermore, it avoids repeatedly performing image recognition to obtain associated image information, thus promoting information compatibility and portability.

[0075] For example, such as Figure 1 The diagram illustrates an optional application scenario of the file processing method provided in this embodiment. The execution end of the file processing method includes an image acquisition device 21, a control device 22, and a decoding device 23. The image acquisition device 21 is used to acquire image data and send it to the control device 22. Taking a landscape image as an example, the image acquisition device 21 can be a digital camera, a smartphone, a camera mounted on a drone, etc. These devices typically possess high-quality lenses, high-resolution image sensors, and advanced image processing technologies, enabling them to capture the delicate textures and rich colors of the landscape. The control device 22 can refer to computer products with communication and storage functions, such as smartphones, desktop computers, laptops, tablets, embedded devices, or other intelligent devices with communication and storage functions. The decoding device 23 can present the image and perform other corresponding operations. It can be a device with a display screen, including but not limited to devices with liquid crystal displays, organic light-emitting diode displays, e-ink screens (such as the screen used in Kindles), such as smartphones, tablets, laptops, desktop computers, smart TVs, etc.

[0076] In an optional embodiment, the image acquisition device 21, the control device 22, and the decoding device 23 are physically separate devices. The image acquisition device 21 is used to capture images. The control device 22 obtains the images and their associated information from the image acquisition device 21, and encapsulates the associated information into the images to obtain the file to be decoded. The decoding device 23 obtains the file to be decoded from the control device 22, parses (decapsulates) the file to be decoded, and obtains the images and their associated information. Based on the associated information, it performs corresponding operations on the images (e.g., rendering).

[0077] It should be noted that, in some alternative embodiments, the image acquisition device 21 and the control device 22 can be a first device that is physically integrated, and the decoding device 23 can be another device that is physically separate from the first device; or, the control device 22 and the decoding device 23 can be a second device that is physically integrated, and the image acquisition device 21 can be another device that is physically separate from the second device; or, the image acquisition device, the control device and the decoding device can be combined to form a third device that is physically integrated.

[0078] It should be noted that, Figure 1 This diagram represents only one possible application scenario for a file processing method. Figure 1 The number of devices included and the names of each device are unlimited, except for... Figure 1 In addition to the devices shown, the application scenarios for file processing methods can also include other devices.

[0079] The application scenarios of the embodiments disclosed herein are not limited. The system architecture and application scenarios described in the embodiments of this disclosure are for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of this disclosure. As those skilled in the art will know, with the evolution of network architecture and the emergence of new service scenarios, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems.

[0080] This disclosure provides a file processing method. For example... Figure 2 As shown, the method includes the following steps:

[0081] S101. Obtain the file to be decoded.

[0082] The file to be decoded includes images and associated information about those images.

[0083] An image can be a single image or one or more frames from a video sequence. An image can also be a portion of a complete image. It is understood that a video itself is composed of a series of consecutive images. The concepts of picture, video, video frame, video image, image, or image region used in the following description can be used interchangeably.

[0084] Image association information is used to describe the feature attributes of an image or a target region within an image. The file to be decoded can be an image / video stream or a media file that includes image association information.

[0085] In some embodiments, the associated information of an image is obtained by performing at least one of the following operations on the image: text recognition, object recognition, or scene recognition.

[0086] In some embodiments, the image association information includes at least one of the following: the type of image association information, the source of image association information, the content of image association information, first indication information, second indication information, and third indication information.

[0087] The first indication information is used to indicate whether the associated information of the image needs to be displayed, the second indication information is used to indicate whether the associated information of the image can be modified (i.e. whether the associated information of the image can be modified by other encoding or decoding units), and the third indication information is used to indicate the language type of the associated information of the image.

[0088] In some embodiments, the types of associated information of an image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information, name information of the image, tag information of the image, and annotation description information of the image.

[0089] In some embodiments, the sources of image association information include creator additions and AI algorithm recognition.

[0090] In some embodiments, the content of the associated information of an image may be human-readable text, or it may be numbers or characters that conform to specific standards, depending on the type of associated information.

[0091] In some embodiments, depending on the content creator's intent, the image's association information can be explicitly displayed in a specific area of ​​the image or implicitly stored as metadata. When the first indication information indicates that the image's association information needs to be displayed, the corresponding association information must be presented simultaneously when the image is rendered.

[0092] In some embodiments, the image association information may also include whether the association information is used to describe multiple consecutive images. Depending on the type and purpose of the association information, the association information may be used to describe a single image or multiple consecutive images.

[0093] In some embodiments, the file to be decoded is obtained by encapsulating the associated information of the image into the image.

[0094] In some embodiments, the file to be decoded is obtained by encapsulating the associated information of the image into the image using an ISO base media file format (ISOBMFF) data box.

[0095] In some embodiments, the file to be decoded is obtained by encapsulating the image's associated information into the image using supplemental enhancement information (SEI) messages.

[0096] In some embodiments, the file to be decoded includes at least one of the following fields: associated information type field, first flag bit, second flag bit, third flag bit, fourth flag bit, fifth flag bit, sixth flag bit, seventh flag bit, eighth flag bit, ninth flag bit, tenth flag bit, eleventh flag bit, twelfth flag bit, thirteenth flag bit, fourteenth flag bit, fifteenth flag bit, and sixteenth flag bit.

[0097] Among them, the association information type field is used to determine the type of association information of the image;

[0098] The first flag is used to indicate whether the associated information of the image is displayed;

[0099] The second flag is used to indicate whether the image's associated information can be modified by any processing device other than the creator;

[0100] The third flag is used to indicate whether the associated information of the image was added manually by the creator or identified by an artificial intelligence algorithm;

[0101] The fourth flag is used to indicate whether the associated information of the image corresponds to a single image or multiple consecutive images;

[0102] The fifth flag is used to indicate the number of associated information entries contained in the associated information attributes of the image;

[0103] The sixth flag is used to determine whether to indicate the language type of the associated information of the image;

[0104] The seventh flag is used to indicate the value of user-defined associated information;

[0105] The eighth flag is used to indicate the length of the language string associated with the image;

[0106] The ninth flag is used to indicate the length of the text string containing the associated information of the image;

[0107] The tenth flag is used to indicate a character in the language string associated with the image;

[0108] The eleventh flag is used to indicate a character in the text string that contains information associated with the image;

[0109] The twelfth flag is used to indicate the length of the object identifier string associated with the image;

[0110] The thirteenth flag is a character in the object identifier string used to indicate the associated information of the image;

[0111] The fourteenth flag is used to indicate the language type of the associated information of the image;

[0112] The fifteenth flag is a string used to indicate the associated information of the image;

[0113] The sixteenth flag is used to indicate the value of user-defined associated information.

[0114] In some embodiments, the indication method of the associated information type field includes at least one of the following: string form, OID form, and numeric form.

[0115] In this way, by storing commonly used image association information in the image processing file, the loss of this information during transmission and processing can be avoided. This allows the image association information to be extracted from the file at any stage of image transmission and processing, saving computational resources required for repetitive processing and ensuring the cross-device and cross-platform application of the image association information. For example, in scenarios involving target localization of large numbers of video surveillance files, the surveillance videos often come from different sources. If these videos already contain image association information (object information), then computational resources and time spent on re-performing AI recognition can be saved, making target localization more cost-effective and faster.

[0116] S102. Parse the file to be decoded to obtain the image and its associated information.

[0117] The method for parsing the file to be decoded to obtain the decoded image can be found in the video codec standard corresponding to the bitstream type, and will not be elaborated here.

[0118] In some embodiments, when the file to be decoded is obtained by encapsulating the image's associated information into the image using ISOBMFF data boxes, the file to be decoded is parsed based on the ISOBMFF data box type.

[0119] S103. Perform corresponding operations on the image based on the image's association information.

[0120] In some embodiments, based on the association information of the image, corresponding operations are performed on the image, including at least one of the following:

[0121] If the first instruction indicates that the associated information of the image needs to be displayed, the image and the associated information of the image are presented;

[0122] If the associated information of an image includes text information within the image, extract the text information within the image;

[0123] When the associated information of an image includes object information within the image, the image is classified and / or quickly located based on the object information within the image.

[0124] When the associated information of an image includes scene information within the image, the content of the image is understood and / or the keywords of the scene within the image are identified based on the scene information within the image.

[0125] If the associated information of an image includes user-defined information, determine whether to process the image and its associated information based on the user-defined information.

[0126] Understandably, when the associated information of an image includes text information within the image, extracting that text information facilitates user operations such as copying or modifying. When the associated information of an image includes user-defined information, the associated information should only be processed if the meaning of the user-defined information is clearly understood. Otherwise, the associated information should be ignored.

[0127] In some embodiments, if the first indication information indicates that the associated information of the image does not need to be displayed, only the image is presented;

[0128] In some embodiments, where the second indication information indicates that the association information of the image can be modified, the user is allowed to adjust the association information of the image as needed while the association information of the image is being displayed.

[0129] In some embodiments, where the second indication indicates that the image's association information is unmodifiable, the image's association information is displayed as read-only, and the user cannot modify it. This may be used to protect the integrity of the image's association information; for example, when the image's association information is automatically generated by the system, it should not be changed by the user.

[0130] In some embodiments, the associated information of an image is displayed based on the language type of the associated information of the image indicated by the third indication information. For example, if the associated information of an image was originally written in English, but the user wants to view the information in Chinese, the third indication information will instruct the system to translate the information into Chinese (if the system supports this function) or mark the original language in some way and provide translation options.

[0131] In some embodiments, where the associated information of an image includes the image's name information, the image is classified and / or the content of the image is understood and / or indexed.

[0132] In some embodiments, where the associated information of an image includes its tag information, the image is indexed and / or categorized.

[0133] Based on this, the associated information of the image is stored together with the image itself, so that the associated information of the image can be seamlessly transmitted and shared between different platforms and devices. This allows different platforms and devices to perform corresponding operations on the image by parsing this associated information, thereby improving image processing efficiency.

[0134] This disclosure provides a file processing method. For example... Figure 3 As shown, the method includes the following steps:

[0135] S201. Obtain the image and its associated information.

[0136] In this context, an image (also known as a source image) can be a single image or one or more frames from a video sequence. The associated information of the image is used to describe the feature attributes of the image or its target region.

[0137] In some embodiments, the image association information is obtained by performing at least one of the following operations on the image: text recognition, object recognition, and scene recognition. There are various related technologies for obtaining image association information through image recognition processing. This disclosure does not limit the use of any image processing technology, and therefore will not elaborate further.

[0138] In some embodiments, the image association information includes at least one of the following: the type of image association information, the source of image association information, the content of image association information, a first indication information, a second indication information, and a third indication information; wherein, the first indication information is used to indicate whether the image association information needs to be displayed, the second indication information is used to indicate whether the image association information can be modified, and the third indication information is used to indicate the language type of the image association information.

[0139] In some embodiments, the types of associated information of an image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information type, image name information, image tag information, and image annotation description information.

[0140] S202. Encapsulate the associated information of the image into the image to obtain the file to be decoded.

[0141] In some embodiments, the associated information of the image is encapsulated into the image using ISOBMFF data boxes to obtain the file to be decoded.

[0142] In some embodiments, the associated information of the image is encapsulated into the image using SEI messages to obtain the file to be decoded.

[0143] S203, Send the file to be decoded.

[0144] In some embodiments, the file to be decoded is sent based on preset rules. For example, the preset rules include a sending period, a start sending time, an end sending time, and a sending time slot.

[0145] For other specific descriptions of S201 to S202, please refer to the descriptions in the above embodiments or examples, which will not be repeated here.

[0146] The following sections describe in detail the implementation process of encapsulating the associated information of an image into the image using different methods.

[0147] Example 1: Image association information is encapsulated into the image using ISOBMFF data boxes. The syntactic structure and semantic information therein can correspond to combinations of various situations described in the above examples or embodiments.

[0148] The following section uses the ISOBMFF data box as an example to illustrate how to indicate the associated information of an image in a picture.

[0149] Image association information attributes, defined as follows:

[0150] Data box type: 'iafp';

[0151] Attribute type: Descriptive item attribute;

[0152] Contains: ItemPropertyContainerBox;

[0153] Mandatory: No;

[0154] Quantity: 0, 1 or more.

[0155] Image association information attributes describe the association information within an image or its target region. The meaning and specific content of the association information are determined by its type. An image or its target region can possess one or more image association information attributes. A single association information attribute can be used to describe one or more images (e.g., animated images).

[0156] grammar:

[0157] aligned(8)class ImageAssociationInfoPropertyextends ItemFullProperty('iafp',version=0,flags=0){

[0158] unsigned int(8)association_info_type;

[0159] unsigned int(1) presentation_flag;

[0160] unsigned int(1) modification_flag;

[0161] unsigned int(1) creation_flag;

[0162] unsigned int(1) consecutive_flag;

[0163] bit(4) reserved;

[0164] unsigned int(8) num_info_entry;

[0165] for(i = 0; i < num_info_entry; i++){

[0166] if(association_info_type < 3 || association_info_type == 5){

[0167] utf8string lang;

[0168] utf8string association_info_text;

[0169] }

[0170] if(association_info_type == 3){

[0171] utf8string OID;

[0172] }

[0173] if((association_info_type == 4){

[0174] unsigned int(16) user_defined_info;

[0175] }

[0176] }

[0177] }

[0178] Semantics:

[0179] When the `association_info_type` field (also known as the association information type field mentioned above) has a value of 0, it indicates that the associated information of the image is text information within the image; a value of 1 indicates that the associated information of the image is scene information within the image, indicated in human-readable string form; a value of 2 indicates that the associated information of the image is object information within the image, indicated in human-readable string form; a value of 3 indicates that the associated information of the image is object information within the image, indicated in OID form, with OID definitions following relevant standards; a value of 4 indicates that the associated information of the image is user-defined information, with the value being an integer, and the meaning of the field value is determined by the user; a value of 5 indicates that the associated information of the image is user-defined information, indicated in human-readable string form, and the meaning of the field value is determined by the user.

[0180] When the presentation_flag (also known as the first flag mentioned above) is 0, it means that the associated information of the image is not displayed; when it is 1, it means that the associated information of the image can be displayed when the image is presented.

[0181] When the modification_flag (also known as the second flag bit mentioned above) is 0, it means that the image's association information cannot be modified by any processing device other than the creator; when it is 1, it means that the image's association information can be modified by any processing device other than the creator.

[0182] When the creation_flag (also known as the third flag bit mentioned above) is 0, it means that the image's association information is added manually by the creator; when it is 1, it means that the image's association information is identified by an AI algorithm.

[0183] When the consecutive_flag (also known as the fourth flag bit mentioned above) is 0, it means that the association information of an image corresponds to a single image; when it is 1, it means that the association information of an image corresponds to multiple consecutive images. When this field is 1, the association information of the corresponding image needs to be associated with an entity group containing multiple images or with multiple consecutive video frame samples.

[0184] num_info_entry (also known as the fifth flag mentioned above) indicates the number of associated information entries contained in the associated information attributes of the current image. When an image's associated information attributes contain multiple associated information entries, these associated information entries indicate mutually substitutable relationships, such as different linguistic expressions of the same information.

[0185] The `lang` field (also known as the sixth flag bit mentioned above) indicates the language type of the image's associated information. This field's indication of the language type should conform to relevant standards.

[0186] association_info_text indicates the text of the association information for the image.

[0187] OID indicates the object identifier, and the value of this field should follow the relevant standard notation.

[0188] The user_defined_info (also known as the seventh flag bit mentioned above) indicates the value of user-defined association information.

[0189] The following is an example of how to indicate the association information of an image in a sample when using an ISOBMFF data box.

[0190] Image association information sample group, defined as:

[0191] Image association information sample groups are used to describe the association information of samples or sample regions. The meaning and specific content of the association information are determined by the type of association information. A sample or a sample region can have one or more association information attributes. An association information attribute can be used to describe one or more images (such as multiple video frames).

[0192] Note: In this technical field, a sample typically corresponds to a single frame of video image in a video.

[0193] grammar:

[0194]

[0195] unsigned int(1)modification_flag;

[0196] unsigned int(1)creation_flag;

[0197] unsigned int(1)consecutive_flag;

[0198] bit(4) is reserved;

[0199] unsigned int(8)num_info_entry;

[0200] for(i=0; i <num_info_entry;i++){

[0201] if(association_info_type<3||association_info_type==5){

[0202] utf8string lang;

[0203] utf8string association_info_text;

[0204] }

[0205] if(association_info_type==3){

[0206] utf8string OID;

[0207] }

[0208] if((association_info_type==4){

[0209] unsigned int(16)user_defined_info;

[0210] }

[0211] }

[0212] }

[0213] The semantics of these fields can be found in the semantic descriptions of the associated information attributes of the images mentioned above, and will not be repeated here.

[0214] The following is an example of how to indicate the associated information of an image in the track when using an ISOBMFF data box.

[0215] Image association information data box, definition:

[0216] Image association information data boxes are used to describe the association information of all samples in a track. The meaning and specific content of the association information are determined by the association information type. A track can contain one or more association information data boxes.

[0217] Note: In this technical field, a track usually refers to a video track, a video track contains one or more samples, and a sample usually corresponds to a frame of video image in the video.

[0218] grammar:

[0219] aligned(8)class ImageAssociationInfoBoxextends Fullbox('iafo',version=0,flags=0){

[0220] unsigned int(8)association_info_type;

[0221]

[0222] The semantics of these fields can be found in the semantic descriptions of the associated information attributes of the images mentioned above, and will not be repeated here.

[0223] Example 2 provides a method for encapsulating image association information into an image using binary bitstream SEI messages. The syntax and semantic information can correspond to combinations of the various cases described in the examples or embodiments above. Specific syntax structures are shown in Table 1.

[0224] Table 1

[0225]

[0226]

[0227] lang_length (also known as the eighth flag mentioned above): indicates the length of the language string associated with the image.

[0228] association_info_text_length (also known as the ninth flag mentioned above): indicates the length of the text string representing the association information of the image.

[0229] char_lang (also known as the tenth flag mentioned above): A character in the language string that indicates the associated information of the image.

[0230] char_association_info_text (also known as the eleventh flag mentioned above): A single character in a text string indicating the association information of the image.

[0231] oid_length (also known as the twelfth flag mentioned above): indicates the length of the OID string that represents the image's associated information.

[0232] char_oid (also known as the thirteenth flag mentioned above): A character in the OID string that indicates the associated information of the image.

[0233] The semantics of the remaining fields in this SEI message can be found in the semantic description of the associated information attributes of the image mentioned above, and will not be repeated here.

[0234] Example 3 provides an extended example based on the existing ISOBMFF data box.

[0235] User description attributes are used to describe the associated information of an image or image region. The meaning and specific content of the associated information are determined by the type of associated information. An image or image region can have one or more associated information attributes. An associated information attribute can be used to describe one or more images (such as animated images).

[0236] grammar:

[0237]

[0238] }

[0239] Semantics:

[0240] The `association_info_type` field (also known as the association information type field) has the following values: 0 indicates text information within the image; 1 indicates scene information within the image, displayed as a human-readable string; 2 indicates object information within the image, displayed as a human-readable string; 3 indicates object information within the image, displayed as an OID (Object ID), with OID definitions following relevant standards; 4 indicates the image name, displayed as a human-readable string; and 5 indicates the image's annotation description. A value of 6 indicates that the image's associated information is its tag information; a value of 7 indicates that the image's associated information is user-defined recognition information of the image content, indicated in human-readable string form, with the meaning of the field value left to the user to determine; a value of 8 indicates that the image's associated information is user-defined understanding information of the image content, indicated in human-readable string form, with the meaning of the field value left to the user to determine; a value of 9 indicates that the image's associated information is user-defined recognition information of the image content, with the value being an integer, and the meaning of the field value left to the user to determine; a value of 10 indicates that the associated information is user-defined understanding information of the image content, with the value being an integer, and the meaning of the field value left to the user to determine.

[0241] When the presentation_flag (also known as the first flag bit mentioned above) is 0, it means that the associated information of the image should not be displayed; when it is 1, it means that the associated information of the image can be displayed when the image is presented.

[0242] When the modification_flag (also known as the second flag bit mentioned above) is 0, it means that the image's association information cannot be modified by any processing device other than the creator; when it is 1, it means that the image's association information can be modified by any processing device other than the creator.

[0243] When the creation_flag (also known as the third flag bit mentioned above) is 0, it means that the image's association information is added manually by the creator; when it is 1, it means that the image's association information is identified by an AI algorithm.

[0244] When the consecutive_flag (also known as the fourth flag bit mentioned above) is 0, it means that the association information of an image corresponds to a single image; when it is 1, it means that the association information of an image corresponds to multiple consecutive images. When this field is 1, the association information of the corresponding image needs to be associated with an entity group containing multiple images or with multiple consecutive video frame samples.

[0245] When `language_flag` (also known as the sixth flag bit mentioned above) is 0, it indicates that the language type does not indicate the association information of the image; when it is 1, it indicates that the language type indicates the association information of the image. When `association_info_type` is 3, `language_flag` can only be 0.

[0246] `num_info_entry` (also known as the fifth flag mentioned above) indicates the number of associated information entries contained in the associated information attribute of an image. When an image's associated information attribute contains multiple associated information entries, these associated information entries indicate mutually substitutable relationships, such as different linguistic expressions of the same information.

[0247] The language type of the associated information for the Lang (i.e., the fourteenth flag bit mentioned above) image should conform to relevant standards (e.g., RFC 5646).

[0248] association_info_string (also known as the fifteenth flag bit mentioned above) is a string containing association information for the image.

[0249] The value of user_defined_info (also known as the sixteenth flag bit mentioned above) is the value of user-defined association information.

[0250] User information sample groups are used to describe the associated information of samples or sample regions. The meaning and specific content of the associated information are determined by the type of associated information. A sample or a sample region can have one or more associated information attributes. An associated information attribute can be used to describe one or more images (such as multiple video frames).

[0251] grammar:

[0252]

[0253] if((association_info_type==9||association_info_type==10){

[0254] unsigned int(16)user_defined_info;

[0255] }

[0256] }

[0257] }

[0258] The semantic part of this user information sample group can be found in the semantic description of the user description attributes, and will not be repeated here.

[0259] The foregoing primarily describes the solutions of the embodiments of this disclosure from a methodological perspective. The following also illustrates a file processing apparatus for executing the file processing methods in any of the above embodiments and their possible implementations. It is understood that, in order to implement the file processing methods, the file processing apparatus includes hardware structures and / or software modules corresponding to the execution of various functions; those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the various examples described in the embodiments of this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0260] This disclosure embodiment can divide the file processing device into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each functional module according to each function.

[0261] Figure 4 This is a file processing apparatus provided in an embodiment of the present disclosure. The file processing apparatus 400 includes: an acquisition module 401, a parsing module 402, and a processing module 403.

[0262] The acquisition module 401 is used to acquire the file to be decoded, which includes an image and the associated information of the image. The associated information of the image is used to describe the feature attributes of the image or the target area of ​​the image.

[0263] The parsing module 402 is used to parse the file to be decoded to obtain the image and its associated information;

[0264] The processing module 403 is used to perform corresponding operations on the image based on the image's association information.

[0265] In some embodiments, the image association information includes at least one of the following: the type of image association information, the source of image association information, the content of image association information, a first indication information, a second indication information, and a third indication information; wherein, the first indication information is used to indicate whether the image association information needs to be displayed, the second indication information is used to indicate whether the image association information can be modified, and the third indication information is used to indicate the language type of the image association information.

[0266] In some embodiments, the types of associated information of an image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information, name information of the image, tag information of the image, and annotation description information of the image.

[0267] In some embodiments, the processing module 403 is specifically used for:

[0268] If the first instruction indicates that the associated information of the image needs to be displayed, the image and the associated information of the image are presented;

[0269] If the associated information of an image includes text information within the image, extract the text information within the image;

[0270] When the associated information of an image includes object information within the image, the image is classified and / or quickly located based on the object information within the image.

[0271] When the associated information of an image includes scene information within the image, the content of the image is understood and / or the keywords of the scene within the image are identified based on the scene information within the image.

[0272] If the associated information of an image includes user-defined information, determine whether to process the image and its associated information based on the user-defined information.

[0273] In some embodiments, the file to be decoded is obtained by encapsulating the image's associated information into the image using ISOBMFF data boxes.

[0274] In some embodiments, the file to be decoded is obtained by encapsulating the image's associated information into the image using SEI messages.

[0275] In some embodiments, the file to be decoded includes at least one of the following fields:

[0276] Associated information type field, first flag bit, second flag bit, third flag bit, fourth flag bit, fifth flag bit, sixth flag bit, seventh flag bit, eighth flag bit, ninth flag bit, tenth flag bit, eleventh flag bit, twelfth flag bit, thirteenth flag bit, fourteenth flag bit, fifteenth flag bit, sixteenth flag bit;

[0277] Among them, the association information type field is used to determine the type of association information of the image;

[0278] The first flag is used to indicate whether the associated information of the image is displayed;

[0279] The second flag is used to indicate whether the image's associated information can be modified by any processing device other than the creator;

[0280] The third flag is used to indicate whether the associated information of the image was added manually by the creator or identified by an artificial intelligence algorithm;

[0281] The fourth flag is used to indicate whether the associated information of the image corresponds to a single image or multiple consecutive images;

[0282] The fifth flag is used to indicate the number of associated information entries contained in the associated information attributes of the image;

[0283] The sixth flag is used to determine whether to indicate the language type of the associated information of the image;

[0284] The seventh flag is used to indicate the value of user-defined associated information;

[0285] The eighth flag is used to indicate the length of the language string associated with the image;

[0286] The ninth flag is used to indicate the length of the text string containing the associated information of the image;

[0287] The tenth flag is used to indicate a character in the language string associated with the image;

[0288] The eleventh flag is used to indicate a character in the text string that contains information associated with the image;

[0289] The twelfth flag is used to indicate the length of the object identifier string associated with the image;

[0290] The thirteenth flag is a character in the object identifier string used to indicate the associated information of the image;

[0291] The fourteenth flag is used to indicate the language type of the associated information of the image;

[0292] The fifteenth flag is a string used to indicate the associated information of the image;

[0293] The sixteenth flag is used to indicate the value of user-defined associated information.

[0294] In some embodiments, the indication method of the associated information type field includes at least one of the following: string form, object identifier form, and numeric form.

[0295] In some embodiments, the associated information of an image is obtained by performing at least one of the following operations on the image: text recognition, object recognition, or scene recognition.

[0296] In some embodiments, the image is a single image or one or more frames from a video sequence.

[0297] For a more detailed description of the acquisition module 401, the parsing module 402, and the processing module 403, as well as a more detailed description of each technical feature therein and a description of the beneficial effects, please refer to the corresponding method embodiment section above, which will not be repeated here.

[0298] Figure 5 This is another file processing apparatus provided in this embodiment. The file processing apparatus 500 includes: an acquisition module 501, an encapsulation module 502, and a communication module 503.

[0299] The acquisition module 501 is used to acquire images and their associated information. The associated information is used to describe the feature attributes of the image or the target area of ​​the image.

[0300] The encapsulation module 502 is used to encapsulate the associated information of the image into the image to obtain the file to be decoded;

[0301] Communication module 503 is used to send the file to be decoded.

[0302] In some embodiments, the image association information includes at least one of the following: the type of image association information, the source of image association information, the content of image association information, a first indication information, a second indication information, and a third indication information; wherein, the first indication information is used to indicate whether the image association information needs to be displayed, the second indication information is used to indicate whether the image association information can be modified, and the third indication information is used to indicate the language type of the image association information.

[0303] In some embodiments, the types of associated information of an image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information type, image name information, image tag information, and image annotation description information.

[0304] In some embodiments, the associated information of an image is obtained by performing at least one of the following operations on the image: text recognition, object recognition, or scene recognition.

[0305] In some embodiments, the image is a single image or one or more frames from a video sequence.

[0306] In some embodiments, the encapsulation module 502 is used to encapsulate the associated information of the image into the image using an ISOBMFF data box to obtain the file to be decoded.

[0307] In some embodiments, the encapsulation module 502 is used to encapsulate the associated information of the image into the image via SEI messages to obtain the file to be decoded.

[0308] For a more detailed description of the acquisition module 501, the encapsulation module 502, and the communication module 503, as well as a more detailed description of their respective technical features and beneficial effects, please refer to the corresponding method embodiment sections above, which will not be repeated here.

[0309] It should be noted that, Figure 4 and Figure 5 Modules in a module can also be called units; for example, a communication module can be called a communication unit. Additionally, in... Figure 4 and Figure 5 In the embodiments shown, the names of the modules may not be the same as those shown in the figures. For example, the communication module may also be called the sending module or the receiving module.

[0310] Figure 4 and Figure 5 If the various units or modules in the present disclosure are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this disclosure. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0311] In implementing the functions of the integrated modules described above using hardware, this disclosure also provides a possible structure for a communication device used to execute the file processing method provided in this disclosure. Figure 6As shown, the communication device 600 includes a communication interface 603, a processor 602, and a bus 604. Optionally, the communication device may also include a memory 601.

[0312] Processor 602 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 602 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 602 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0313] Communication interface 603 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0314] The memory 601 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0315] In one possible implementation, the memory 601 can exist independently of the processor 602. The memory 601 can be connected to the processor 602 via a bus 604 and is used to store instructions or program code. When the processor 602 calls and executes the instructions or program code stored in the memory 601, it can implement the file processing method provided in this embodiment of the disclosure.

[0316] In another possible implementation, the memory 601 can also be integrated with the processor 602.

[0317] Bus 604 can be an extended industry standard architecture (EISA) bus, etc. Bus 604 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0318] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform a file processing method as described in any of the above embodiments.

[0319] In one exemplary embodiment, the computer may be the aforementioned file processing device, and this disclosure does not limit the specific form of the computer.

[0320] In some examples, the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0321] This disclosure provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the file processing method described in any of the above embodiments.

[0322] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A file processing method, characterized in that, The method includes: Obtain the file to be decoded, which includes an image and associated information of the image. The associated information of the image is used to describe the feature attributes of the image or a target area of ​​the image. The file to be decoded is parsed to obtain the image and its associated information; Based on the association information of the image, perform corresponding operations on the image.

2. The method according to claim 1, characterized in that, The image association information includes at least one of the following: the type of the image association information, the source of the image association information, the content of the image association information, a first indication information, a second indication information, and a third indication information; wherein, the first indication information is used to indicate whether the image association information needs to be displayed, the second indication information is used to indicate whether the image association information can be modified, and the third indication information is used to indicate the language type of the image association information.

3. The method according to claim 2, characterized in that, The types of associated information of the image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information, name information of the image, tag information of the image, and annotation description information of the image.

4. The method according to claim 3, characterized in that, The operation on the image based on the association information of the image includes at least one of the following: When the first indication information indicates that the associated information of the image needs to be displayed, the image and the associated information of the image are presented; If the associated information of the image includes text information within the image, extract the text information within the image; If the associated information of the image includes object information within the image, the image is classified and / or quickly located based on the object information within the image. When the associated information of the image includes scene information within the image, the content of the image is understood and / or the keywords of the scene within the image are identified based on the scene information within the image; If the image's associated information includes the user-defined information, determine whether to process the image and its associated information based on the user-defined information.

5. The method according to claim 1, characterized in that, The file to be decoded is obtained by encapsulating the associated information of the image into the image using the ISOBMFF data box format of the International Organization for Standardization Basic Media File Format.

6. The method according to claim 1, characterized in that, The file to be decoded is obtained by encapsulating the associated information of the image into the image using supplementary enhancement information (SEI) messages.

7. The method according to claim 5 or 6, characterized in that, The file to be decoded includes at least one of the following fields: Associated information type field, first flag bit, second flag bit, third flag bit, fourth flag bit, fifth flag bit, sixth flag bit, seventh flag bit, eighth flag bit, ninth flag bit, tenth flag bit, eleventh flag bit, twelfth flag bit, thirteenth flag bit, fourteenth flag bit, fifteenth flag bit, sixteenth flag bit; The associated information type field is used to determine the type of associated information of the image; The first flag is used to indicate whether the associated information of the image is displayed; The second flag is used to indicate whether the associated information of the image can be modified by any processing device other than the creator; The third flag is used to indicate whether the associated information of the image is added manually by the creator or identified by an artificial intelligence algorithm; The fourth flag bit is used to indicate whether the associated information of the image corresponds to a single image or multiple consecutive images; The fifth flag bit is used to indicate the number of associated information entries contained in the associated information attributes of the image; The sixth flag is used to determine whether to indicate the language type of the associated information of the image; The seventh flag bit is used to indicate the value of user-defined associated information; The eighth flag bit is used to indicate the length of the language string associated with the image; The ninth flag bit is used to indicate the length of the text string associated with the image; The tenth flag is used to indicate a character in the language string associated with the image; The eleventh flag is used to indicate a character in the text string associated with the image; The twelfth flag bit is used to indicate the length of the object identifier string of the associated information of the image; The thirteenth flag bit is used to indicate a character in the object identifier string of the associated information of the image; The fourteenth flag bit is used to indicate the language type of the associated information of the image; The fifteenth flag bit is used to indicate a string of associated information about the image; The sixteenth flag bit is used to indicate the value of user-defined associated information.

8. The method according to claim 7, characterized in that, The indication method of the associated information type field includes at least one of the following: string form, object identifier form, and numerical form.

9. The method according to claim 1, characterized in that, The associated information of the image is obtained by performing at least one of the following operations on the image: text recognition, object recognition, and scene recognition.

10. The method according to claim 9, characterized in that, The image is a single image or one or more frames from a video sequence.

11. A file processing method, characterized in that, The method includes: Obtain the image and its association information, wherein the association information is used to describe the image or the feature attributes within the target region of the image; The associated information of the image is encapsulated into the image to obtain the file to be decoded; Send the file to be decoded.

12. The method according to claim 11, characterized in that, The image association information includes at least one of the following: the type of the image association information, the source of the image association information, the content of the image association information, a first indication information, a second indication information, and a third indication information; wherein, the first indication information is used to indicate whether the image association information needs to be displayed, the second indication information is used to indicate whether the image association information can be modified, and the third indication information is used to indicate the language type of the image association information.

13. The method according to claim 12, characterized in that, The types of associated information of the image include at least one of the following: text information within the image, object information within the image, scene information within the image, user-defined information type, name information of the image, tag information of the image, and annotation description information of the image.

14. The method according to claim 11, characterized in that, The associated information of the image is obtained by performing at least one of the following operations on the image: text recognition, object recognition, and scene recognition.

15. The method according to claim 11, characterized in that, The image is a single image or one or more frames from a video sequence.

16. The method according to claim 11, characterized in that, The step of encapsulating the associated information of the image into the image to obtain the file to be decoded includes: The associated information of the image is encapsulated into the image using ISOBMFF data boxes to obtain the file to be decoded.

17. The method according to claim 11, characterized in that, The step of encapsulating the associated information of the image into the image to obtain the file to be decoded includes: The associated information of the image is encapsulated into the image using SEI messages to obtain the file to be decoded.

18. A communication device, characterized in that, include: Memory and processor; Memory and processor are coupled; The memory is used to store instructions that can be executed by the processor; When the processor executes the instructions, it performs the method as described in any one of claims 1 to 17.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a communication device, cause the communication device to perform the method as described in any one of claims 1 to 17.

20. A computer program product, characterized in that, When the computer program product is executed, it implements the method as described in any one of claims 1 to 17.

Images