Photographing method, apparatus, electronic device, and computer program product

Abstract

Embodiments of the present application are applicable to the technical field of terminal and image processing, and provide a photographing method and device, electronic equipment and computer program product. By applying the method, the electronic equipment can display a first preview image collected by a camera on a first screen after responding to a first photographing instruction. Then, by determining label information matched with the first preview image, the electronic equipment can label the first preview image, so as to display a second preview image containing the label information on the first screen. In this way, in the photographing process, the electronic equipment can automatically generate and display personalized label information without manual input or editing by the user. Moreover, by automatically determining the label information matched with the first preview image, the accuracy of the label information can be improved, the user can accurately record the content of the photo, the time cost of later photo arrangement is reduced, and the user operation is simplified.

Description

Technical Field

[0001] This application relates to the fields of terminal and image processing technology, and in particular to a photographing method, apparatus, electronic device, and computer program product. Background Technology

[0002] Users can take photos using the camera on their electronic devices. For example, users can take photos using their mobile phone camera. For the captured images, users can also record information such as the time and location of the photo, or descriptive information about the image's background and context, to reflect the story and emotions behind the image.

[0003] In existing technologies, users can annotate images after taking them using the editing functions provided by electronic devices, thereby recording relevant information. However, this post-capture annotation method requires users to manually input and edit annotation information, increasing the user's operational cost. Furthermore, if users fail to annotate the images promptly after taking them, the accuracy of the final annotation information may be affected by user forgetfulness. Summary of the Invention

[0004] This application provides a photographing method, apparatus, electronic device, and computer program product that can automatically generate and display personalized annotation information during the photographing process, eliminating the need for manual input or editing by the user. Furthermore, by automatically determining the annotation information that matches the image, the accuracy of the annotation information can be improved, facilitating accurate recording of image content by the user, reducing post-processing time costs, and simplifying user operation.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] The first aspect of this application provides a method for taking pictures, including:

[0007] In response to the first photo-taking command, the first preview image captured by the camera is displayed on the first screen;

[0008] Determine the annotation information that matches the first preview image, the annotation information including numerical information;

[0009] Based on the first preview image and the annotation information, a second preview image containing the annotation information is generated on the first screen.

[0010] By applying the photographing method provided in this application, the electronic device can automatically generate and display personalized annotation information including digital information during the photographing preview stage, so that the user can preview the annotation effect when taking the picture. This makes it convenient for the user to obtain photos or images with annotation information without having to manually input or edit the corresponding annotation information after taking the picture, reducing the time cost for the user to organize the photos later and simplifying the user operation.

[0011] In one possible implementation of the first aspect of this application, determining the annotation information matching the first preview image includes: identifying the image content contained in the first preview image; and determining the annotation information matching the first preview image based on the image content.

[0012] In this way, electronic devices can determine personalized annotations by recognizing and analyzing image content, so that the annotation information can fully reflect the image content and ensure the accuracy of the annotation information.

[0013] In one example, determining the annotation information matching the first preview image based on the image content may include: determining at least one object feature of the image content, which may be a feature corresponding to an object included in the image content; and generating annotation information to characterize the at least one object feature based on the at least one object feature.

[0014] In this way, electronic devices can identify objects in an image and generate standard information that matches the actual characteristics of those objects, ensuring that the standard information matches the image content.

[0015] In another example, determining the annotation information matching the first preview image based on the image content may also include: retrieving at least one data source to obtain retrieval information related to the image content; and generating annotation information matching the first preview image based on the retrieval information. The data source may include at least one of the following: calendar information, user information, and gallery information.

[0016] In this way, electronic devices can identify and analyze image content, and combine this with relevant retrieved information to determine personalized annotations, thereby further improving the accuracy of the annotation information.

[0017] In another example, determining the annotation information matching the first preview image based on the image content may further include: determining the current user behavior based on the image content; and generating annotation information adapted to the user behavior based on the matching relationship between the current user behavior and the user's historical behavior.

[0018] Electronic devices can generate annotation information by recognizing and analyzing image content and combining it with the matching relationship between user behavior and user history behavior. This allows the annotation information to fully reflect the user behavior corresponding to the image content.

[0019] In another possible implementation of the first aspect of this application, determining the annotation information matching the first preview image based on the image content may further include: determining historical images associated with the image content based on the image content; and determining digital information for annotating the first preview image based on the annotation information of the historical images. The digital information for annotating the first preview image and the digital information contained in the annotation information of the historical images may be continuous.

[0020] This application embodiment generates annotation information containing digital information, which enables continuity between annotated images. This helps users obtain real records and feedback in specific behaviors such as persisting in exercise and learning, facilitating continuous accumulation and contributing to the development of long-term good habits.

[0021] In another possible implementation of the first aspect of the embodiments of this application, determining the annotation information matching the first preview image based on the image content may further include: determining the style of the annotation information based on the image content; and generating the annotation information matching the first preview image according to the style.

[0022] It should be understood that, in addition to generating personalized annotation information, electronic devices can also automatically adjust the annotation format according to the overall style of the image content, ensuring that the final annotation information is visually harmonious with the image content and enhancing the image's aesthetic appeal.

[0023] In one possible implementation of the first aspect of this application, generating a second preview image containing annotation information on a first screen based on a first preview image and annotation information includes: determining an annotation area of ​​the annotation information in the first preview image; and displaying the annotation information in the annotation area to generate a second preview image containing the annotation information on the first screen.

[0024] Specifically, determining the annotation area of ​​the annotation information in the first preview image includes: determining the layout information of the image content contained in the first preview image; and determining the annotation area of ​​the annotation information based on the layout information.

[0025] In this way, electronic devices can determine the appropriate location for the generated annotation information based on the actual layout of the image content, avoiding arbitrary annotations that could negatively impact the image's aesthetics.

[0026] In one possible implementation of the first aspect of this application, the electronic device may be a device including multiple screens, such as a first screen and a second screen. Therefore, after generating a second preview image containing annotation information on the first screen, the process may further include: generating a third preview image based on the second preview image; and displaying the third preview image on the second screen.

[0027] In one example, the electronic device including a first screen and a second screen can be a foldable screen electronic device. Therefore, displaying a third preview image on the second screen may include: determining the screen size of the second screen of the foldable screen electronic device; and generating a third preview image adapted to the screen size of the second screen based on the second preview image.

[0028] In this way, by generating a third preview image that matches the second screen, the best visual effect can be obtained when the third preview image is displayed on the second screen. Electronic devices can adaptively generate appropriate annotation information according to the device form, ensuring that users can have a good user experience in different device forms.

[0029] In this embodiment of the application, after generating a second preview image containing annotation information on a first screen based on a first preview image and annotation information, the electronic device can respond to a second photo-taking command to perform a photo-taking operation on the second preview image and obtain a camera image with the same content as the second preview image.

[0030] A second aspect of this application provides a photographing device, including a camera, a processing unit, and a first screen; wherein:

[0031] The first screen is used to display a first preview image captured by the camera in response to a first photo capture command;

[0032] The processing unit is configured to determine annotation information matching the first preview image, and generate a second preview image containing the annotation information based on the first preview image and the annotation information, wherein the annotation information includes numerical information;

[0033] The first screen is also used to display the second preview image.

[0034] A third aspect of this application provides a photographing device, including a first preview image display module, a labeling information determination module, and a second preview image display module; wherein:

[0035] The first preview image display module is used to display the first preview image captured by the camera on the first screen in response to the first photo capture command;

[0036] The annotation information determination module is used to determine annotation information that matches the first preview image, wherein the annotation information includes numerical information;

[0037] The second preview image display module is used to generate a second preview image containing the annotation information on the first screen based on the first preview image and the annotation information.

[0038] A fourth aspect of this application provides an electronic device that may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the photographing method described in the first aspect above.

[0039] A fifth aspect of this application provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned method steps to achieve the first aspect described above.

[0040] A sixth aspect of this application provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the photographing method described in the first aspect. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a photographing method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the structure of an electronic device to which the photographing method provided in this application embodiment is applicable;

[0043] Figure 3 This is a schematic diagram of the overall process of a photographing method provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of a photographing method provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram illustrating the screen display effect of an electronic device according to an embodiment of this application;

[0046] Figure 6 This is a schematic diagram illustrating another electronic device screen display effect provided in an embodiment of this application;

[0047] Figure 7 This is a structural block diagram of a photographing device provided in an embodiment of this application. Detailed Implementation

[0048] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0049] The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0050] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0051] The steps involved in the photographing method, apparatus, electronic device, and computer program product provided in this application are merely examples. Not all steps are mandatory, nor are all contents within each step required. They can be added or removed as needed during use. The same step or steps or contents with the same function in the embodiments of this application can be referenced and learned from each other in different embodiments.

[0052] When a user takes a photo using an electronic device, a preview image is displayed on the screen. This preview image is the actual image captured by the device's camera. In other words, the preview image displayed on the screen during photo capture is identical to the image actually captured by the camera. Users can adjust various shooting parameters based on the preview image, such as shooting angle and exposure, and then take the photo to obtain an image identical to the preview. In some scenarios, users may need to annotate the image. For example, they could annotate the scene depicted in the image or the user behavior represented by the image. In one possible implementation, image annotation requires editing the image using the electronic device or application's editing functions after it has been captured by the camera, thus adding the corresponding information. This operation increases the user's operational cost because it requires manual input and editing of the annotation information.

[0053] To address the aforementioned issues, this application provides a photographing method, apparatus, electronic device, and computer program product. By applying the above method, when taking a photograph using an electronic device, the device can automatically recognize the content of the preview image displayed on the screen, thereby determining the information that can be annotated, and annotating the preview image with this information. This allows users to directly obtain an image with annotated information by performing a photographing operation, reducing the subsequent manual input or editing of annotation information and improving the efficiency of image annotation.

[0054] like Figure 1 The diagram shown is a schematic representation of a photographing method provided in an embodiment of this application. According to... Figure 1 As shown in the process, when a user takes a picture with an electronic device, the screen of the electronic device can display the current preview image 101. The preview image 101 can be an image directly captured by the camera and displayed on the screen in real time after the electronic device responds to the user's operation or command.

[0055] In this embodiment of the application, the electronic device can identify the preview image 101 displayed in real time. For example, after identification, in Figure 1 The preview image 101 shown includes a user 101a wearing a birthday hat 101b. The electronic device can determine annotation information matching the preview image 101 based on the identified information. For example, the electronic device can determine that the current scene is a birthday celebration scene based on the user 101a and birthday hat 101b identified from the preview image 101, and the corresponding annotation information matching the preview image 101 can be birthday-related information. Based on this, the electronic device can further determine the age of user 101a. For example, the electronic device can determine that user 101a is 29 years old based on calendar information, user information, etc. Therefore, the annotation information determined by the electronic device that matches the preview image 101 can be related information used to annotate user 101a celebrating their 29th birthday. In one possible implementation of this application embodiment, the annotation information determined by the electronic device can include numerical information. For example, in the aforementioned example, the determined annotation information can be "Celebrating a 29th Birthday" or "29" containing the number "29". th Information such as "HAPPY BIRTHDAY" was sent.

[0056] During the process of determining the annotation information, the electronic device can further determine the annotation area, which can be the location or area in the preview image 101 used to annotate the determined annotation information. For example, as... Figure 1As shown, the labeled area in preview image 101 can be area 101c associated with birthday hat 101b. Since birthday hat 101b in preview image 101 is image content related to birthday celebration, in one possible implementation, area 101c for labeling can be an image area located near birthday hat 101b.

[0057] In one possible implementation of the embodiments of this application, such as Figure 1 As shown, the electronic device can also determine the style of the annotation information, which can match the annotation information to be annotated. For example, in Figure 1 The preview image 101 shown indicates that in the case of a birthday celebration, the style of the annotation information can also be a style related to birthdays, festivals, etc.

[0058] After determining the annotation information, annotation area, and annotation style, the electronic device can annotate the information in the corresponding area according to the determined style. For example, Figure 1 As shown, the determined annotation information is marked in region 101c of the preview image 101, thereby obtaining... Figure 1 The preview image 102 shown can be considered as obtained by annotating the preview image 101. Preview image 102 includes the annotation information "29" automatically generated by the electronic device through image recognition processing. th The content of "HAPPY BIRTHDAY".

[0059] In one possible implementation of this application's embodiments, the electronic device may include multiple screens, depending on its type. For example, a foldable screen electronic device may include an inner screen and an outer screen. Therefore, when applying the method provided in this application's embodiments, the image containing annotation information displayed in real time can also be adaptively adjusted according to the different forms of the electronic device. For example, as... Figure 1 As shown, when the electronic device is a flat-screen electronic device with only one screen, the aforementioned preview image 102 can be displayed on that single screen. When the electronic device is a foldable-screen electronic device with an inner screen and an outer screen, the aforementioned preview image 102 can be displayed on the inner screen of the foldable-screen electronic device. Furthermore, the foldable-screen electronic device can determine the image that can be displayed on the outer screen based on the preview image 102 displayed on the inner screen and the size of the outer screen. This image can be obtained by cropping the preview image 102 based on the outer screen size, or it can be obtained by processing the annotation information in the preview image 102 based on the outer screen size. For example, Figure 1 Image 103 shown may be an image for display on the outer screen of a foldable electronic device, and image 103 is a representation of the annotation information "29" in preview image 102.th The image obtained after processing "HAPPY BIRTHDAY" has the same style. In this way, for foldable electronic devices with multiple screens, it can be ensured that the preview images displayed on each screen have the same style.

[0060] The electronic devices applicable to the photographing method provided in this application can be mobile phones, tablets, or other devices with photographing or video recording functions. This application does not limit the type of electronic device.

[0061] For example, Figure 2 A schematic diagram of an electronic device 200 is shown. The structure of the above-described electronic device can be referenced from... Figure 2 The structure of the electronic device 200.

[0062] like Figure 2 As shown, the electronic device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0063] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 200. In some embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0064] Processor 210 may include one or more processing units. For example, processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0065] The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions. For example, when the electronic device 200 receives a user's photo-taking instruction, the controller can activate the camera 293 to take a picture and display the image content captured by the camera 293 in real time on the display screen 294 as a preview image.

[0066] In some embodiments of this application, the processor 210 and the camera 293 can communicate via a camera serial interface (CSI) to enable the electronic device 200 to perform the shooting function. The processor 210 and the display screen 294 can communicate via a display serial interface (DSI) to enable the electronic device 200 to perform the display function.

[0067] In this embodiment, the display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments of this application, the electronic device 200 may include one or N display screens 294, where N is a positive integer greater than 1.

[0068] Camera 293 is used to capture still images, moving images, or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments of this application, the electronic device 200 may include one or N cameras 293, where N is a positive integer greater than 1.

[0069] The photographing method provided in this application can be implemented on an electronic device with the above-described hardware structure. The following describes the photographing method provided in this application in detail using an electronic device with the above-described hardware structure as an example.

[0070] like Figure 3 The diagram shown is a schematic representation of the overall process of a photographing method provided in an embodiment of this application. Figure 3 This illustrates the process of taking a picture using the photographing method provided in the embodiments of this application, and automatically annotating the image during the photographing process.

[0071] S301, Enable automatic annotation function for photos.

[0072] The automatic image annotation function refers to the function of an electronic device automatically recognizing the content of the image to be captured and annotating it based on the recognized content when taking a picture. The image capture method provided in this application embodiment is executed only when the automatic image annotation function is enabled. Otherwise, if the automatic image annotation function is disabled, the electronic device will take a picture in the conventional way, and the captured image will not include annotation information.

[0073] In one possible implementation of this application, the automatic photo annotation function can be manually enabled by the user through the system settings entry provided by the electronic device on the corresponding page; or, the function can be enabled on the camera settings page after entering the camera application.

[0074] In another possible implementation of this application, in addition to the camera application, other applications in the electronic device can also provide the photo-taking function. For example, system applications or third-party applications in the electronic device. These applications will invoke the camera application when a photo needs to be taken. Therefore, the automatic photo annotation function can also be manually enabled by the user in the settings page when other applications invoke the camera application to take photos.

[0075] S302. Analyze and identify user behavior based on static or dynamic image content.

[0076] In the embodiments of this application, the static or dynamic image can be a preview image displayed on the screen when the electronic device takes a picture. For example, the first preview image can refer to an image captured in real time by the camera or webcam after the camera of the electronic device is activated.

[0077] The first preview image can be a static image or a dynamic image; this application does not limit this.

[0078] For example, in response to a first photo-taking command, the electronic device can activate the camera, capture images in real time using the camera or webcam, and display a first preview image on the screen. The first photo-taking command could be an instruction to activate the camera, triggering the electronic device to enter photo-taking mode. With automatic photo annotation enabled, the electronic device can analyze and identify the static or dynamic first preview image to confirm the current user behavior. Subsequent automatic photo annotation functions implemented by the electronic device can be based on the current user behavior. For example, the electronic device can annotate the image based on the confirmed current user behavior.

[0079] In this embodiment, the electronic device can analyze and identify the first preview image to determine the image content, and then determine the current user behavior based on the image content. That is, the electronic device can identify the image content contained in the first preview image and determine the current user behavior based on the identified image content. This user behavior can be the behavior corresponding to the image content presented in the first preview image.

[0080] Depending on whether the first preview image contains the user or other users, the electronic device may employ different methods to determine the current user behavior based on the image content. For example, such as... Figure 1 The first preview image shown when a user takes a selfie indicates that the user's behavior is celebrating a birthday; or, the first preview image shown when a user lifts a barbell indicates that the user's behavior could be exercising, and so on. These user behaviors can be determined based on the actions or behaviors performed by the user in the preview image, when the preview image includes the user or other users.

[0081] In another possible implementation of this application embodiment, when the preview image does not include a user, the electronic device can analyze the image content in the preview image and determine the current user behavior based on the image content itself. For example, after identifying the image content contained in the first preview image, the electronic device can determine the object features of the image content. These object features can be features corresponding to objects included in the image. For example, the objects included in the image content can be mountains, steps, etc. Based on the features of these objects, the electronic device determines that the current user behavior can be mountain climbing. As another example, if the objects included in the image content are cakes, candles, etc., the electronic device can determine that the current user behavior can be celebrating a birthday based on the features of these objects.

[0082] In another possible implementation of this application, when the electronic device determines the current user behavior based on the image content itself, it can also determine the current user behavior by obtaining retrieval information related to the image content through retrieval of one or more data sources in the electronic device.

[0083] For example, the electronic device recognizes that the image content in the first preview image includes items such as a cake. By retrieving the data source, it can determine that the day is the user's birthday. Combining the recognized image content, the electronic device can determine that the current user behavior is celebrating a birthday. The aforementioned data source may include calendar information, gallery information, or user information stored on the electronic device.

[0084] In the aforementioned example, the electronic device can determine the current user behavior based on the image content contained in the identified first preview image and at least one object feature of the image content; it can also determine the current user behavior by combining the identified image content with retrieval information related to the image content through retrieval of at least one data source; or, the electronic device can further use at least one object feature of the image content and the retrieved retrieval information together to determine the current user behavior, thereby improving the accuracy of user behavior determination.

[0085] The user behaviors identified through the various methods described above can be used to determine the specific content to be labeled.

[0086] S303. Determine the annotation format.

[0087] In this embodiment of the application, the annotation format can be a type of information used to represent the specific type of annotation information. For example, the annotation format can include various types such as count type, time type, or description type.

[0088] Among them, the count-based annotation information can indicate information such as the number of times the image content in the first preview image is displayed. For example, in a fitness scenario, the count-based annotation information can be used to mark the number of times a user has checked in for a workout, such as the annotation information "13th workout this month".

[0089] Time-related annotations can include the time the image was taken, or specific times corresponding to the image content. For example, in birthday celebration scenarios, image annotations could include information reflecting the user's birthday or age.

[0090] Descriptive annotations can include information that specifically describes or introduces the image content or the user behavior reflected in the image content. For example, annotations describing the weather or air quality of the day.

[0091] In this embodiment, the annotation format can be preset based on user preferences. For example, the user can preset to use count-based annotation information to annotate subsequently captured images, and the electronic device can preferentially use the count-based format to annotate the images. Alternatively, the electronic device can also annotate the currently captured images using the same format as the previous annotation each time an annotation is needed. For example, if the user used a descriptive format for the previous annotation, the electronic device can preferentially use the same descriptive format when automatically annotating the current image. Alternatively, the electronic device can provide the user with multiple options through the screen, allowing the user to select in real time to determine the specific annotation format.

[0092] In one possible implementation of this application, the annotation format can be determined based on the electronic device's classification of the image content in the first preview image. For example, after identifying the image content in the first preview image, the electronic device can classify the image content to determine the specific annotation format. In one example, the electronic device can pre-store a mapping table between different categories of image content and their corresponding annotation formats. After determining the category of the image content, the electronic device can determine the annotation format of the current category of image content by consulting the mapping table. In another example, the electronic device can determine the annotation format of the current image by matching the annotation formats of other annotated images in the image library. For example, if the electronic device identifies the first preview image as an image from a fitness scene, and confirms that most of the annotated images from other fitness scenes in the image library use a count-based annotation format, then the electronic device can determine that the annotation format of the image in this fitness scene is a count-based annotation. This application does not limit how the annotation format is determined.

[0093] S304. Search historical data to confirm the specific content of the annotation.

[0094] In this embodiment, the specific content of the annotation can be the specific information ultimately used for annotation, i.e., the content of the annotation information. The electronic device can determine the specific content of the annotation by combining the determined user behavior and the annotation format. For example, in images of birthday celebration scenes, assuming the determined user behavior is celebrating a birthday and the determined annotation format is time-based, the specific content of the annotation could be "celebrate".

[0095] In one possible implementation of this application, the electronic device can determine the specific content to be labeled by combining historical data. For example... Figure 3 As shown, historical data can be obtained by searching one or more data sources such as calendar information, user information, and / or gallery information. Determining the specific content of the annotation by searching historical data can reflect the continuity of the annotation information. For example, in some fitness scenarios, by searching historical data, the number of times a user exercises can be determined, thereby determining the specific content of the annotation, such as "the 13th exercise this month" as in the previous example.

[0096] S305. Determine appropriate annotation locations based on the image layout.

[0097] In this embodiment, the annotation location can refer to the image location used to annotate the annotation content determined in the aforementioned steps. This image location can be a certain image area in the preview image. The electronic device can determine a suitable annotation location based on the layout of the identified image content.

[0098] In one possible implementation of this application, a suitable annotation location can be a blank area in the image. For example, the electronic device identifies various objects contained in the image content, such as people, objects, or scenes, and determines the annotation location for areas in the preview image that are not used to display these objects. In this way, the annotated image will not affect the display of the objects in the original image due to the annotation information.

[0099] In another possible implementation of this application, the electronic device can determine the position of the object represented by the annotation information in the preview image based on image layout information, and determine the associated region of the object's location as a suitable annotation position or annotation region. The associated region can be the area surrounding or within a certain range of the object represented by the annotation information. In this way, the annotation information can be associated with the object it represents.

[0100] S306. Determine the annotation style based on the overall image style, the specific content of the annotation, and the user's historical data.

[0101] Annotation style can be the interface display style that reflects the final display effect of the annotated information. For example, annotation style can include flat style, bright style, shadow or texture style, festive style, etc.

[0102] In this embodiment, the electronic device can determine the annotation style based on the overall style of the preview image and the specific content of the annotation, so that the annotation style matches the image and ensures a harmonious and unified visual effect. For example, in the birthday celebration image in the aforementioned example, a festive style can be used, so that the image and annotation information present a festive visual effect.

[0103] Alternatively, electronic devices can combine historical data with the overall style of the preview image and the specific content of the annotations.

[0104] In one possible implementation of this application, the electronic device can pre-configure multiple annotation styles and pre-associate different image content or user behavior with the annotation styles. After determining the graphic content or user behavior, the annotation style to be adopted can be quickly determined based on the pre-configured association relationships.

[0105] S307. The way the view is presented is determined according to the device configuration.

[0106] Device form factor refers to the external structural form of an electronic device. The form factor is determined by the device type; different types of devices have different form factors. For example, foldable screen electronic devices and flat screen electronic devices belong to different device types, and their form factors are also different.

[0107] In this application embodiment, the presentation method of the view varies depending on the device form. The aforementioned view may refer to a preview image with annotation information. For example, a second preview image corresponding to a first preview image captured and presented in real time by a camera or webcam may be an image presented to the user as a preview image after the annotation information has been determined and actually annotated according to the aforementioned steps.

[0108] Taking a flat-screen electronic device as an example, such devices typically consist of only one screen on the front. After activating the camera, this screen can be used to display the image captured in real-time by the camera or webcam—the first preview image. After determining the annotation information according to the aforementioned steps, the electronic device can generate a second preview image based on the first preview image and the annotation information, and display the second preview image on this screen. The second preview image is essentially a preview image with the annotation information.

[0109] Compared to flat-screen electronic devices, foldable-screen electronic devices may include more screens. For example, a foldable-screen electronic device may include an outer screen and an inner screen. In one example, the foldable-screen electronic device may display the first preview image from the aforementioned example on the inner screen. Accordingly, the foldable-screen electronic device may display a second preview image labeled with annotation information on the inner screen. Furthermore, the foldable-screen electronic device may also generate a third preview image based on the second preview image and display the third preview image on the outer screen.

[0110] In another example, the foldable screen electronic device can display the first preview image from the aforementioned example on the outer screen. Accordingly, the foldable screen electronic device can display a second preview image labeled with annotation information on the outer screen, and a third preview image generated based on the second preview image on the inner screen.

[0111] In this embodiment, the electronic device can process the second preview image according to the size of the screen used to display the third preview image, thereby obtaining a third preview image adapted to that screen size. For example, the electronic device can crop or scale the second preview image according to the screen size to obtain a third preview image adapted to that size. In one example, the electronic device can crop or scale the second preview image based on annotation information, so that the third preview image obtained after cropping or scaling includes complete annotation information.

[0112] S308. Generate the corresponding view in the appropriate location based on the annotation content, annotation location, and annotation style.

[0113] In this embodiment, the view generated by the electronic device based on the annotation content, annotation location, and annotation style can be a preview image during the photo-taking process. For example, the aforementioned second preview image, third preview image, etc.

[0114] Based on the displayed preview image with annotation information, the electronic device can respond to the photo-taking command and perform a photo-taking operation on the preview image to obtain a camera image with the same content as the preview image, which is automatically annotated with the annotation information determined according to the aforementioned steps.

[0115] By applying the photographing method provided in this application, the electronic device can automatically generate and display personalized annotation information during the photographing preview stage. This allows users to preview the annotation effect while taking the picture, making it convenient for them to obtain photos or images with annotation information without having to manually input or edit the corresponding annotation information after taking the picture. This reduces the time cost for users to organize photos later and simplifies user operation. Secondly, the electronic device can determine personalized annotation content by recognizing and analyzing the image content and combining it with historical data. This ensures that the annotation information can fully reflect the image content or the user behavior corresponding to the image content, guaranteeing the accuracy of the annotation information. On this basis, the electronic device can also automatically adjust the annotation form and annotation position according to the overall style of the image content, ensuring that the final annotation information is visually harmonious with the image content and enhancing the aesthetics of the image. Thirdly, automatic annotation ensures the authenticity of the annotation information, allowing users to obtain real records and feedback on specific behaviors such as persisting in exercise and learning. This facilitates continuous accumulation by users and helps them develop good long-term habits. Fourthly, the electronic device can adaptively generate appropriate annotation information according to the device form, ensuring that users can obtain a good user experience under different device forms.

[0116] based on Figure 3 The overall process shown in this application embodiment provides a photographing method, such as... Figure 4 As shown, the method includes the following steps S401-S403:

[0117] S401, In response to the first photographing command, display the first preview image captured by the camera on the first screen.

[0118] The first photo-taking command can be a command to activate the camera of the electronic device. Once activated, the camera is operational, capable of capturing images in real time and displaying the captured first preview image on the first screen. For electronic devices with only one screen, such as flat-screen devices, the first screen is the only screen. For electronic devices with multiple screens, such as foldable devices, the first screen can be the main screen when taking photos. Foldable devices can also include a second screen. For example, the first screen can be the inner screen of the foldable device, and the second screen can be the outer screen. Alternatively, the first screen can be the outer screen of the foldable device, and the second screen can be the inner screen.

[0119] like Figure 5 and Figure 6 The figures shown are schematic diagrams illustrating the screen display effects of two electronic devices provided in embodiments of this application. Figure 5 In (a), an example is shown where a first preview image 501 is displayed on the first screen of an electronic device. Figure 6 In (a), an example is shown where a first preview image 601 is displayed on the first screen of an electronic device. Figure 5 The first preview image 501 in (a) and Figure 6 The first preview image 601 in (a) are all images captured in real time by the electronic device through the camera in response to the first photo-taking command. Figure 6 The first preview image 601 in (a) is an image captured when the user takes a selfie with the electronic device in hand, i.e., a selfie image.

[0120] S402. Determine the annotation information that matches the first preview image, the annotation information including numerical information.

[0121] Electronic devices can identify and analyze a first preview image to determine annotation information that matches the first preview image. Specifically, the electronic device can identify the image content contained in the first preview image and determine the annotation information that matches the first preview image based on the image content. For example, the electronic device can target... Figure 5 The first preview image 501 in (a) Figure 6 Image recognition analysis is performed on the first preview image 601 in (a) to obtain information on what image content is contained in the first preview image 501 and the first preview image 601.

[0122] In one possible implementation of this application, the electronic device can determine at least one object feature of the image content, and generate annotation information to characterize the at least one object feature based on the at least one object feature. The object feature may be a feature corresponding to an object included in the image content.

[0123] For example, such as Figure 5 As shown, electronic devices can recognize Figure 5 The image content of the first preview image 501 in (a) is used to determine the objects included in the image content, such as the user 501a, the birthday hat 501b, and the birthday candle 501c. Then, based on the characteristics of these objects, corresponding annotation information is generated. For example, based on the characteristics of objects such as the birthday hat 501b and the birthday candle 501c, annotation information related to celebrating a birthday can be generated.

[0124] In one example, the electronic device identified Figure 5 After understanding the objects and their corresponding characteristics included in (a) of the diagram, the current user behavior can also be determined based on these objects and their characteristics. For example, based on... Figure 5 The objects and features of the first preview image 501 shown in (a) can confirm that the current user behavior is celebrating a birthday, thereby generating corresponding annotation information based on the user behavior.

[0125] In another possible implementation of this application, when the electronic device determines the annotation information matching the first preview image based on the image content, it can also retrieve at least one data source to obtain retrieval information related to the image content, thereby generating annotation information matching the first preview image based on the retrieval information. The aforementioned data source may include at least one of the following: calendar information, user information, gallery information, etc.

[0126] For example, such as Figure 5 As shown, after identifying the image content contained in the first preview image 501, the electronic device can retrieve data sources such as calendar information and user information to obtain information related to the contained image content. For example, if the image content of the first preview image 501 includes user 501a, a birthday hat 501b, and birthday candles 501c, the electronic device can obtain the user's birthday information by retrieving calendar information and / or user information. Based on this, combined with the current time, the electronic device can confirm that the day is the user's birthday and thus obtain annotation information related to celebrating the birthday.

[0127] In another possible implementation of this application, the electronic device can determine the current user behavior based on the identified image content, and generate annotation information adapted to the current user behavior based on the matching relationship between the current user behavior and the user's historical behavior.

[0128] For example, such as Figure 6 As shown, based on the recognition obtained Figure 6 The image content contained in the first preview image 601 shown in (a) is, for example... Figure 6 In (a) of the example, a user is taking a selfie in an indoor space with multiple fitness equipment using an electronic device. Therefore, the electronic device can determine that the current user behavior is fitness exercise. Based on this, the electronic device can generate annotation information adapted to the current user behavior based on the matching relationship between this behavior and the user's historical behaviors.

[0129] In this embodiment of the application, the annotation information generated by the electronic device may include numerical information, which can be used to represent the number of times a user's behavior occurs, the time of such behavior, and other numerical information that can describe user behavior or object characteristics. Examples include the number of times and the time of fitness exercises, the number of times and the time of studying, etc.

[0130] Therefore, in one example, the electronic device can determine historical images associated with the image content identified from the first preview image, and then determine the digital information used to annotate the first preview image based on the annotation information of the historical images.

[0131] For example, historical images that are related to the image content can be obtained by searching data sources such as image libraries. For example, in Figure 6 In (b), a gallery page 603 is shown, which stores a photo 603a labeled "7th workout check-in." This photo 603a indicates that the user's historical activity was also related to fitness exercises. Thus, the electronic device can generate annotation information adapted to the current user behavior based on the matching relationship between the user behavior confirmed from the first preview image 601 and the user's historical behavior confirmed from the photo 603a, obtaining annotation information that characterizes the user behavior in the first preview image 601.

[0132] In this embodiment of the application, the digital information used to annotate the first preview image and the digital information contained in the annotation information of the historical image can be continuous. For example, the annotation information of the historical image, namely photo 603a stored in the gallery page 603, is "7th fitness check-in". When determining the annotation information that can be used to annotate the user behavior in the first preview image 601, the information to be annotated can be determined as "8th fitness check-in" based on the continuity of the digital information.

[0133] As can be seen from the foregoing examples, when an electronic device determines the annotation information matching the first preview image based on the image content contained in the first preview image obtained through recognition, it can generate annotation information that can be used to characterize at least one object feature by determining that the image content has at least one object feature; alternatively, it can retrieve retrieval information related to the image content by searching at least one data source, and then generate annotation information matching the first preview image based on the retrieval information; or, it can determine the current user behavior based on the image content, and generate annotation information adapted to the current user behavior based on the matching relationship between the current user behavior and the user's historical behavior. The above three different methods for determining annotation information can be used individually or simultaneously.

[0134] For example, an electronic device can combine object features and retrieval information obtained from a data source to determine annotation information; or, it can combine object features and user behavior to determine annotation information; or, it can combine retrieval information obtained from a data source and user behavior to determine annotation information; or it can combine object features, retrieval information, and user behavior together to determine annotation information. This application does not limit this approach.

[0135] When determining annotation information using any one or more of the three methods described above, the electronic device can add numerical information to the annotation information to obtain annotation information containing numerical information. For example, Figure 5 The annotation information containing the number "29" shown in (b) is "29". th"HAPPY BIRTHDAY", and Figure 6 The label information containing the number "8" shown in (c) reads "8th gym check-in".

[0136] In one possible implementation of this application, when the electronic device determines the annotation information, it can also determine the style of the annotation information based on the image content, and generate annotation information matching the first preview image according to the determined style.

[0137] S403. Based on the first preview image and the annotation information, generate a second preview image containing the aforementioned annotation information on the first screen.

[0138] In this embodiment, after determining the annotation information, the electronic device also needs to determine the annotation area of ​​the annotation information in the first preview image. Thus, by displaying the annotation information in the annotation area, a second preview image containing the annotation information can be generated on the first screen of the electronic device. Specifically, the electronic device can determine the layout information of the image content contained in the first preview image, and determine the annotation area of ​​the annotation information based on the layout information.

[0139] For example, such as Figure 5 As shown in (b), the electronic device can determine the area suitable for annotation as the area near the birthday hat 501b based on the layout of the image content, and then display the annotation information in that area to obtain the annotation information "29". th The second preview image 502 of "HAPPY BIRTHDAY". For example... Figure 6 As shown in (c), the electronic device can obtain a second preview image 602 containing the annotation information "8th fitness check-in" by determining the annotation area based on the layout of the image content.

[0140] In one possible implementation of this application embodiment, the electronic device may further include multiple screens. For example, Figure 5 The electronic devices shown and Figure 6 The electronic device shown includes a first screen and a second screen. Figure 5 In (a) and (b) of the above, the first screen can be used to display the first preview image 501 and the second preview image 502; Figure 6 In (a) and (c) above, the first screen can be used to display a first preview image 601 and a second preview image 602. The electronic device described above can generate a third preview image based on the second preview image and display the third preview image on the second screen. For example, the electronic device can be based on... Figure 5 The second preview image 502 in (b) generates a third preview image and displays it on the second screen.

[0141] In one example, the electronic device comprising multiple screens can be a foldable screen electronic device. For example, the aforementioned first and second screens are screens of the foldable screen electronic device. Therefore, when generating a third preview image based on a second preview image, the screen size of the second screen of the foldable screen electronic device can be determined, and a third preview image adapted to the screen size of the second screen can be generated based on the second preview image.

[0142] In this embodiment, the second preview image generated by the electronic device, containing annotation information, is an image from the photo preview stage, not an actual captured image. Therefore, when the electronic device displays the second preview image on its screen, it can respond to a second photo-taking command to perform a photo-taking operation on the second preview image, obtaining a camera image with the same content as the second preview image. The aforementioned second photo-taking command can be a user-initiated command via gesture or voice, instructing the electronic device to perform a photo-taking operation. For example, in... Figure 6 In (c) of the above, the user can issue a command to the electronic device to perform a photo-taking operation by clicking the camera button 602b on the screen. In response to this command, the electronic device can capture a photo of the second preview image, obtaining a camera image with the same content as the second preview image. For example, by... Figure 5 By capturing the second preview image 502 in (b) of the above, a camera image with the same content as the second preview image 502 can be obtained; by... Figure 6 By capturing the second preview image 602 in (c) of the image, a camera image with the same content as the second preview image 602 can be obtained. This camera image, once stored, becomes the... Figure 6 Photo 603b is stored in gallery page 603 as shown in (d) of the image.

[0143] Combination Figure 5 and Figure 6 As shown, the photography method provided in this application embodiment can be applied to both selfie scenarios and non-selfie scenarios. For example, Figure 5 The image shown illustrates the changes in the screen preview image during the taking of a photo by an electronic device in a non-selfie scenario. Figure 6 The diagram illustrates the changes in the screen preview image during the selfie process of an electronic device taking a picture. Therefore, the photo-taking method provided in this application can be applied to selfie scenarios, such as taking a selfie in front of a mirror, using a front-facing camera, or using a rear-facing camera, etc. This application does not limit the application scenarios of the aforementioned embodiments.

[0144] This application embodiment can divide an electronic device into functional modules based on the above method example. For example, each function can be divided into a separate functional module, or one or more functions can be integrated into a single functional module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation. The following description uses the example of dividing each function into a separate functional module.

[0145] Corresponding to the above embodiments, refer to Figure 7 This diagram illustrates a structural block diagram of a photographing device provided in an embodiment of this application. This device can be applied to the electronic devices described in the foregoing embodiments. Specifically, the device may include the following modules: a first preview image display module 701, a labeling information determination module 702, and a second preview image display module 703, wherein:

[0146] The first preview image display module 701 is used to display the first preview image captured by the camera on the first screen in response to the first photo capture command;

[0147] The annotation information determination module 702 is used to determine annotation information that matches the first preview image, wherein the annotation information includes numerical information;

[0148] The second preview image display module 703 is used to generate a second preview image containing the annotation information on the first screen based on the first preview image and the annotation information.

[0149] Corresponding to the above embodiments, this application also provides a photographing device, including a camera, a processing unit, and a first screen. For example, the device may be an electronic device including a camera, a processing unit, and a first screen as described in the foregoing embodiments. Wherein:

[0150] The first screen is used to display a first preview image captured by the camera in response to a first photo capture command;

[0151] The processing unit is configured to determine annotation information matching the first preview image, and generate a second preview image containing the annotation information based on the first preview image and the annotation information, wherein the annotation information includes numerical information;

[0152] The first screen is also used to display the second preview image.

[0153] In one possible implementation of this application embodiment, the device may further include a second screen, such as an electronic device including a second screen. Exemplarily, the electronic device including the second screen may be a foldable screen electronic device. Wherein:

[0154] The processing unit is further configured to: generate a third preview image based on the second preview image;

[0155] The second screen is used to display the third preview image.

[0156] The aforementioned device may be an electronic device in the foregoing embodiments, or it may be a unit or component in the aforementioned electronic device that can perform the corresponding function.

[0157] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0158] This application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the photographing methods described in the foregoing embodiments.

[0159] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned related method steps to achieve the photographing method in the various embodiments described above.

[0160] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the photographing methods in the various embodiments described above.

[0161] This application also provides a chip, which can be a processor, or the chip includes a processor. The processor can be a general-purpose processor or a dedicated processor; wherein, the processor is used to support the foldable screen device in performing the above-mentioned related steps to implement the photography methods in the foregoing embodiments.

[0162] Optionally, the chip also includes a transceiver for receiving control from the processor to support the electronic device in performing the aforementioned steps to implement the photographing method in the various embodiments described above.

[0163] Optionally, the chip may also include a storage medium.

[0164] The chip can be implemented using one or more field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.

[0165] Finally, it should be noted that the above description is only a specific implementation of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the protection scope of this application.

Claims

1. A method for taking photos, characterized in that, include: In response to the first photo-taking command, the first preview image captured by the camera is displayed on the first screen; Determine annotation information that matches the first preview image, the annotation information including numerical information; Based on the first preview image and the annotation information, a second preview image containing the annotation information is generated on the first screen.

2. The method according to claim 1, characterized in that, The step of determining the annotation information that matches the first preview image includes: Identify the image content contained in the first preview image; Based on the image content, determine the annotation information that matches the first preview image.

3. The method according to claim 2, characterized in that, The step of determining the annotation information matching the first preview image based on the image content includes: The image content is determined to have at least one object feature, wherein the object feature is a feature corresponding to an object included in the image content; Generate annotation information to characterize at least one of the object features based on at least one of the object features.

4. The method according to claim 2, characterized in that, The step of determining the annotation information matching the first preview image based on the image content includes: Retrieve at least one data source to obtain retrieval information related to the image content; Based on the search information, annotation information matching the first preview image is generated.

5. The method according to claim 4, characterized in that, The data source includes at least one of the following: Calendar information, user information, and gallery information.

6. The method according to claim 2, characterized in that, The step of determining the annotation information matching the first preview image based on the image content includes: Determine the current user behavior based on the image content; Based on the matching relationship between the current user behavior and the user's historical behavior, annotation information adapted to the user behavior is generated.

7. The method according to any one of claims 2 to 6, characterized in that, The step of determining the annotation information matching the first preview image based on the image content further includes: Based on the image content, identify historical images that are associated with the image content; Based on the annotation information of the historical images, digital information for annotating the first preview image is determined.

8. The method according to claim 7, characterized in that, The digital information used to annotate the first preview image is continuous with the digital information contained in the annotation information of the historical image.

9. The method according to any one of claims 2 to 8, characterized in that, The step of determining the annotation information matching the first preview image based on the image content further includes: Determine the style of the annotation information based on the image content; Generate annotation information matching the first preview image according to the style described.

10. The method according to any one of claims 1 to 9, characterized in that, The step of generating a second preview image containing the annotation information on the first screen based on the first preview image and the annotation information includes: Determine the annotation area of ​​the annotation information in the first preview image; The annotation information is displayed in the annotation area to generate a second preview image containing the annotation information on the first screen.

11. The method according to claim 10, characterized in that, Determining the annotation area of ​​the annotation information in the first preview image includes: Determine the layout information of the image content contained in the first preview image; The annotation area of ​​the annotation information is determined based on the layout information.

12. The method according to claim 10 or 11, characterized in that, Also includes: A third preview image is generated based on the second preview image; The third preview image is displayed on the second screen.

13. The method according to claim 12, characterized in that, The first screen and the second screen are the screens of a foldable electronic device. The step of generating a third preview image based on the second preview image includes: Determine the screen size of the second screen of the foldable screen electronic device; A third preview image is generated based on the second preview image, adapted to the screen size of the second screen.

14. The method according to any one of claims 1 to 13, characterized in that, After generating a second preview image containing the annotation information on the screen based on the first preview image and the annotation information, the method further includes: In response to the second photo-taking command, a photo-taking operation is performed on the second preview image to obtain a camera image with the same content as the second preview image.

15. A photographing device, characterized in that, Includes a camera, a processing unit, and a first screen; wherein: The first screen is used to display a first preview image captured by the camera in response to a first photo capture command; The processing unit is configured to determine annotation information matching the first preview image, and generate a second preview image containing the annotation information based on the first preview image and the annotation information, wherein the annotation information includes numerical information; The first screen is also used to display the second preview image.

16. The apparatus according to claim 15, characterized in that, It also includes a second screen; in which: The processing unit is further configured to: generate a third preview image based on the second preview image; The second screen is used to display the third preview image.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 14.

Images