File processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a file processing method and device, electronic equipment and a storage medium, and belongs to the technical field of computers.The method comprises the steps that a first multimedia file and barrier-free configuration information are obtained, and the barrier-free configuration information comprises at least one of user barrier type information, user requirement information and user requirement information; the user demand information is used for representing an output mode demand of a user for a multimedia file; based on the first multimedia file and the barrier-free configuration information, a second multimedia file is obtained, and the second multimedia file is used for outputting the content of the first multimedia file in at least one mode.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to a file processing method, apparatus, electronic device and storage medium. Background Technology

[0002] For users with color vision impairment, their ability to see or distinguish colors is poor, which presents certain difficulties in daily life. For example, people with red-green color blindness cannot accurately distinguish the colors of traffic lights. Therefore, related technologies can use auxiliary information such as graphics, textures, and text to help users obtain information.

[0003] However, in the above methods, since the addition of auxiliary information needs to be done in the early stages of color general design, it is not compatible with and can not be used for existing multimedia files such as pictures and videos. As a result, it is difficult for users to obtain information from existing multimedia files. Consequently, electronic devices are not flexible enough in their processing of multimedia files, and the processing effect is poor. Summary of the Invention

[0004] The purpose of this application is to provide a file processing method, apparatus, electronic device, and storage medium that can enhance the diversity of multimedia file processing methods of electronic devices and improve the file processing effect of electronic devices.

[0005] In a first aspect, embodiments of this application provide a file processing method, which includes: obtaining a first multimedia file and accessibility configuration information, the accessibility configuration information including at least one of the following: user disability type information and user demand information, the user demand information being used to characterize the user's output method requirements for the multimedia file; and obtaining a second multimedia file based on the first multimedia file and the accessibility configuration information, the second multimedia file being used to output the content of the first multimedia file in at least one manner.

[0006] Secondly, embodiments of this application provide a file processing apparatus, which includes: an acquisition module and a processing module; the acquisition module is used to acquire a first multimedia file and accessibility configuration information, the accessibility configuration information including at least one of the following: user disability type information and user demand information, the user demand information being used to characterize the user's output method requirements for the multimedia file; the processing module is used to obtain a second multimedia file based on the first multimedia file and accessibility configuration information acquired by the acquisition module, the second multimedia file being used to output the content of the first multimedia file in at least one way.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program / program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In this embodiment, a first multimedia file and accessibility configuration information are obtained. The accessibility configuration information includes at least one of the following: user disability type information and user requirement information. The user requirement information characterizes the user's output method requirements for the multimedia file. Based on the first multimedia file and the accessibility configuration information, a second multimedia file is obtained. This second multimedia file is used to output the content of the first multimedia file in at least one way. In this solution, when the user needs to obtain the information contained in the first multimedia file, the electronic device can obtain the second multimedia file based on the first multimedia file and the accessibility configuration information. That is, the electronic device can reconstruct the first multimedia file by combining the accessibility configuration information, so that the final second multimedia file can meet the user's feedback requirements. In other words, the electronic device can display the information content contained in the first multimedia file to the user in the form of user preferences or requirements. Thus, the user can more conveniently and accurately obtain the information of the first multimedia file. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device. Attached Figure Description

[0012] Figure 1 This is one of the flowcharts of a file processing method provided in the embodiments of this application;

[0013] Figure 2 This is a second flowchart of a file processing method provided in an embodiment of this application;

[0014] Figure 3AThis is one of the schematic diagrams of a document processing procedure provided in the embodiments of this application;

[0015] Figure 3B This is a second schematic diagram of a document processing flowchart provided in an embodiment of this application;

[0016] Figure 3C This is the third schematic diagram of a document processing process provided in the embodiments of this application;

[0017] Figure 4 This is the fourth schematic diagram of a document processing flowchart provided in the embodiments of this application;

[0018] Figure 5 This is the fifth schematic diagram of a document processing flowchart provided in the embodiments of this application;

[0019] Figure 6A This is one of the schematic diagrams of a color confusion line provided in the embodiments of this application;

[0020] Figure 6B This is a second schematic diagram of a color-mixing line provided in the embodiments of this application;

[0021] Figure 7 This is the third flowchart of a document processing method provided in the embodiments of this application;

[0022] Figure 8 This is a schematic diagram of an example of a settings interface provided in an embodiment of this application;

[0023] Figure 9 This is a schematic diagram of the structure of a document processing device provided in an embodiment of this application;

[0024] Figure 10 This is one of the hardware structure diagrams of an electronic device provided in the embodiments of this application;

[0025] Figure 11 This is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0027] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects. For example, a first object can be one or more, where "more" means at least two. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0028] The terms "at least one" and "at least one of" in this application's specification refer to any one, any two, or a combination of two or more of the included objects. For example, "at least one of a, b, and c" can mean "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple, and multiple means at least two. Similarly, "at least two" means two or more, and its meaning is similar to "at least one". The identifiers in this application are text, symbols, images, etc., used to indicate information, and can use controls or other containers as carriers for displaying information, including but not limited to text identifiers, symbol identifiers, and image identifiers.

[0029] The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. The terminology involved in the embodiments of this application is explained below.

[0030] Color vision deficiency refers to an individual's abnormal ability to distinguish colors, making it impossible to differentiate between different colors like a normal person. This is not a simple vision problem, but rather caused by a variety of factors interfering with the visual system's color perception. It can be divided into two categories: color blindness and color weakness.

[0031] Color blindness: Patients completely lose the ability to distinguish colors. Based on the types of colors they cannot distinguish, color blindness is further divided into red-green color blindness, green-green color blindness, blue-yellow color blindness, and total color blindness. Red-green color blindness is the most common; also known as Daltonism, patients have difficulty distinguishing between red and green, and these two colors appear similar in hue to them. Total color blindness is even rarer; patients can only see a black, white, and gray world and are completely unable to perceive color. Patients with blue-yellow color blindness have difficulty distinguishing between blue and green, purple and red, and yellow and pink.

[0032] Color weakness: Patients with color weakness have a reduced ability to distinguish colors. They are not completely unable to distinguish colors, but rather need specific conditions to accurately differentiate them. Color weakness includes total color weakness and partial color weakness, such as red weakness, green weakness, and blue-yellow weakness.

[0033] Color Universal Design is a comprehensive design concept aimed at enabling everyone, regardless of age, gender, cultural background, or visual ability, to easily perceive and understand color information. It can assist in design for those with color vision deficiencies and cataracts, aiming to select colors that are equally effective for color-sensitive individuals as for sighted people. Common methods include adding auxiliary textures or using universally recognized color codes to make the design independent of color or to enhance contrast.

[0034] Hearing impairment (dysaudia) refers to organic or functional abnormalities in the various levels of nerve centers in the auditory system responsible for sound transmission, perception, and the comprehensive analysis of sound, resulting in varying degrees of hearing loss. It is commonly referred to as deafness. Only severe hearing loss is termed deafness, characterized by the patient's inability to hear any speech in both ears. Hearing loss that does not reach this level of severity is called hearing loss.

[0035] The diffusion model is a deep learning model based on probabilistic generation. Its core principle is to simulate the random process of gradually adding noise and denoising data, that is, the random process of forward diffusion and reverse reconstruction. By using neural networks to learn the data distribution pattern, it can ultimately generate high-quality samples from random noise.

[0036] Artificial Intelligence Generated Content (AIGC) models are artificial intelligence (AI) systems that generate original content such as text, images, audio, and video through machine learning techniques. Their core technologies include Generative Adversarial Networks (GANs) and diffusion models.

[0037] A text encoder is a core component in Natural Language Processing (NLP) and cross-modal tasks. Its role is to convert text data into machine-understandable numerical representations, such as vectors or tensors, providing basic features for downstream tasks such as text classification, machine translation, and image generation.

[0038] Variational Autoencoders (VAEs) are generative models that combine deep learning and probabilistic graphical models. They can learn the latent distribution of data, enabling them not only to reconstruct input data but also to generate new data samples. VAEs introduce the concept of probabilistic distributions into the traditional autoencoder (AE) model. Their core objective is to learn the latent distribution of data and make it follow a specific prior distribution, such as the standard normal distribution, thereby achieving data generation.

[0039] Latent space, a concept originating from mathematics and machine learning, refers to a low-dimensional, structured abstract space used to represent the core features of high-dimensional, complex data. The low-dimensional representation of latent space primarily aims to reduce information loss and improve computational efficiency. In high-dimensional data, through mapping and compression, the data is transformed into low-dimensional vectors, allowing for more efficient feature extraction and processing. For example, when using VAEs, the low-dimensional representation of latent variables helps visualize and understand the structure of the data. Furthermore, the low-dimensional representation enables a better capture of the essential characteristics of complex data.

[0040] U-Net is an image segmentation model based on Convolutional Neural Networks (CNN). Its core design is a symmetrical encoder-decoder structure that fuses multi-scale features through skip connections. It is suitable for high-precision pixel-level tasks such as medical image segmentation and satellite image analysis.

[0041] Downsampling is a fundamental operation in signal and image processing, referring to the process of reducing the amount of data by lowering the data sampling rate or resolution. In image processing and deep learning, downsampling is often used to reduce the spatial resolution of an image while preserving key feature information.

[0042] QKV: In attention mechanisms, QKV refers to three parameters: Query, Key, and Value. These three parameters play a crucial role in attention mechanisms, dynamically capturing the relationships between elements in the input sequence.

[0043] Specifically, Q represents the target for which attention needs to be calculated (such as a word), signifying a "question" or "requirement". K represents each element in the input sequence, used to calculate similarity with the Query, similar to an "index" or "tag". V stores the actual information, and the final attention output is a weighted sum of the Values, representing the "actual content".

[0044] Confusion line: Mark these points on the CIE chromaticity chart and draw a line through them. You will find that this line connects all the colors that people with color vision deficiencies find difficult to distinguish.

[0045] The CIE chromaticity diagram, established by the International Commission on Illumination (CIE) in 1931, is a standard color representation system. It is a two-dimensional diagram drawn using chromaticity coordinates x and y to represent the chromaticity characteristics of colors, namely hue and saturation, but without luminance information. The CIE diagram is based on the CIE XYZ color space, which was derived through a series of human visual experiments and aims to provide a device-independent color representation system.

[0046] Controls: Elements in a graphical user interface that can receive user input to perform corresponding processing or display relevant data. Controls can include, but are not limited to, virtual buttons, sliders, progress bars, and checkboxes.

[0047] Interface: Refers to the medium through which users interact with electronic devices. The interface allows users to send commands to the system via input devices and receive feedback information via output devices. Input devices can be keyboards, mice, touchscreens, etc.; monitors, speakers, etc.

[0048] For individuals with color vision deficiency, their ability to see or distinguish colors is impaired, leading to difficulties in daily life, such as buying fruit, choosing clothing, and identifying the colors of traffic lights. Specifically, individuals with red-green color blindness cannot accurately distinguish the colors of traffic lights or differentiate between red and green apples by color.

[0049] Therefore, in the early stages of universal color design, accessibility design can be added, such as adding auxiliary information like graphics, textures, and text, so that users can obtain accurate information without relying solely on color.

[0050] For example, in the initial design of traffic lights, the red light was designed as a triangle, the yellow light as a square, and the green light as a circle. By adding different shapes to different colors, users can better identify traffic lights.

[0051] However, in the color universal design of the aforementioned related technologies, because auxiliary information needs to be added in the early stage of color universal design, it is not possible to achieve compatibility and utilization with existing multimedia files such as pictures and videos. For example, colorblind people cannot watch existing old movies and old photos normally. That is, the related technologies cannot meet the user's barrier-free access to information in existing multimedia files, and the processing method of multimedia files is not flexible enough to meet the user's needs. As a result, the file processing effect of electronic devices is poor.

[0052] The document processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0053] The file processing method provided in this application can be applied to scenarios where users view multimedia files. The following examples illustrate the file processing method provided in this application.

[0054] Scenario 1: When a user with color vision impairment needs to view traffic lights, the electronic device can acquire a multimedia file containing traffic lights, such as a real-time recorded video. The electronic device can also acquire accessibility configuration information related to the user. Based on the real-time recorded video and accessibility configuration information, the electronic device can generate and output a second multimedia file, such as a video with added text descriptions. The user can then learn the color of the traffic lights through the output video.

[0055] Scenario 2: During the viewing of a movie by a user with color vision impairment, the electronic device can acquire a first multimedia file, such as the movie the user is watching, and the electronic device can acquire accessibility configuration information related to the user. Then, based on the movie and the accessibility configuration information, the electronic device can generate and output a second multimedia file, such as a movie with added text descriptions, so that the user can learn about the colors in each frame of the movie through the output movie.

[0056] It should be noted that the above scenarios 1 and 2 are merely exemplary examples of some scenarios that may be applied to the embodiments of this application. In actual implementation, the embodiments of this application can also be applied to any possible scenarios such as users with color vision impairment or hearing impairment needing to obtain information from multimedia files. The embodiments of this application are not limited here.

[0057] Based on the above-mentioned scenario applied in the embodiments of this application, the file processing method provided in the embodiments of this application obtains a first multimedia file and accessibility configuration information. The accessibility configuration information includes at least one of the following: user disability type information and user demand information. The user demand information is used to characterize the user's output method requirements for the multimedia file. Based on the first multimedia file and the accessibility configuration information, a second multimedia file is obtained. The second multimedia file is used to output the content of the first multimedia file in at least one way. In this solution, when the user needs to obtain the information contained in the first multimedia file, the electronic device can obtain the second multimedia file based on the first multimedia file and the accessibility configuration information. That is, the electronic device can combine the accessibility configuration information to reconstruct the first multimedia file so that the final obtained second multimedia file can meet the user's feedback requirements. In other words, the electronic device can display the information content contained in the first multimedia file to the user in the form of user preferences or requirements. Thus, the user can more conveniently and accurately obtain the information of the first multimedia file. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device.

[0058] The file processing method provided in this application is executed by a file processing device, which can be an electronic device, or a functional module or entity within an electronic device. This application does not limit the specific implementation of this method. The following will use an electronic device as an example to illustrate the file processing method provided in this application.

[0059] This application provides a document processing method. Figure 1 A flowchart illustrating a file processing method provided in an embodiment of this application is shown. Figure 1 As shown, the document processing method provided in this application embodiment may include the following steps 201 and 202.

[0060] Step 201: The electronic device acquires the first multimedia file and accessibility configuration information.

[0061] In some embodiments of this application, the first multimedia file described above may include, but is not limited to, at least one of the following: images, audio, and video.

[0062] In some embodiments of this application, the aforementioned images may include, but are not limited to, at least one of the following: images stored by an electronic device, images viewed by a user through an electronic device, and preview images displayed by a camera application of an electronic device.

[0063] In some embodiments of this application, the above-mentioned image can be any one of a still image, a moving image, or a moving photograph.

[0064] In some embodiments of this application, the audio may include, but is not limited to, at least one of the following: audio stored in an electronic device, audio acquired by the electronic device through a network connection, and audio recorded in real time by the electronic device.

[0065] In some embodiments of this application, the aforementioned video may include, but is not limited to, at least one of the following: video stored in an electronic device, video acquired by the electronic device through a network connection, and video stream recorded in real time by the electronic device.

[0066] In some embodiments of this application, when a user views or listens to a first multimedia file, the electronic device can receive user input to trigger subsequent steps. For example, the input can be a click on a special control displayed on the interface, a press on a physical button on the electronic device, or any feasible input set by the user.

[0067] In this embodiment of the application, the accessibility configuration information may include at least one of the following: user disability type information and user requirement information.

[0068] In some embodiments of this application, the aforementioned user impairment type information may represent the type of color vision impairment or hearing impairment that the user has.

[0069] For example, the aforementioned color vision deficiencies can include two categories: color blindness and color weakness. Color blindness can include, but is not limited to, any of the following: red-green color blindness, blue-yellow color blindness, or total color blindness. Color weakness can include, but is not limited to, any of the following: mild amblyopia or severe amblyopia.

[0070] In this embodiment of the application, the above-mentioned user requirement information can be used to characterize the user's requirements for the output method of multimedia files.

[0071] In some embodiments of this application, the above output method may include, but is not limited to, at least one of the following: displaying the content of a multimedia file through subtitles, playing the content of a multimedia file through audio, and providing haptic feedback on the intensity of a multimedia file.

[0072] In some embodiments of this application, the aforementioned user requirement information may further include user requirement information regarding the environmental context. This environmental context may include, but is not limited to, at least one of the following: the device capabilities of the electronic device, and the network status of the network used by the electronic device.

[0073] For example, the aforementioned device capabilities may include, but are not limited to, at least one of the following: whether the electronic device has a screen, whether the electronic device has a speaker, and whether the electronic device has a tactile device.

[0074] In some embodiments of this application, the accessibility configuration information described above may be information that is actively set by the user in advance.

[0075] In some embodiments of this application, the electronic device can learn the modal combination of the multimedia file to be generated, as well as the similarity, focus, and expression of the multimedia file output by the modal combination, based on the above-mentioned accessibility configuration information.

[0076] For example, the above modal combinations may include, but are not limited to, at least one of the following: images, captions, and audio. For example, images and audio, images and captions, or images, audio, and captions.

[0077] For example, if the user has color vision deficiency, the accessibility configuration information set by the user may be more biased towards obtaining the content of the first multimedia file through hearing; if the user has hearing impairment, the accessibility configuration information set by the user may be more biased towards obtaining the content of the first media file through vision.

[0078] It should be noted that the method for users to set the above accessibility configuration information can be found in the relevant descriptions of steps 301 to 303 below, and will not be repeated here in the embodiments of this application.

[0079] Step 202: The electronic device obtains a second multimedia file based on the first multimedia file and accessibility configuration information.

[0080] In some embodiments of this device, combined with Figure 1 ,like Figure 2 As shown, step 202 above can be specifically achieved through steps 202a to 202d.

[0081] Step 202a: The electronic device inputs the first multimedia file and accessibility configuration information into the target model to obtain the first feature information.

[0082] In this embodiment of the application, the first feature information includes the description information and key features of the first multimedia file.

[0083] In some embodiments of this application, the target model can be a preset AIGC diffusion model, or a model that is further trained or adjusted based on the preset AIGC diffusion model.

[0084] In some embodiments of this application, when the electronic device obtains the first multimedia file and accessibility configuration information, the electronic device can directly input the first multimedia file and accessibility configuration information into the target model, or the electronic device can preprocess the first multimedia file and accessibility configuration information first, and then input the preprocessed content into the target model.

[0085] For example, when the first multimedia file is a static image, a dynamic video frame sequence, or a real-time video stream, the preprocessing performed on the first multimedia file by the electronic device may include, but is not limited to, at least one of the following: resolution adjustment, color space conversion, keyframe extraction, noise removal, and initial screening of target detection.

[0086] In some embodiments of this application, before step 202a above, the file processing method provided in the embodiments of this application may also include the following steps 401 and 402, and the "electronic device inputs the first multimedia file and accessibility configuration information into the target model" in step 202 above can be specifically implemented by the following step 202a1.

[0087] Step 401: The electronic device encodes the accessibility configuration information using a text encoder to obtain text feature information.

[0088] It is understandable that electronic devices can use text encoders to convert accessibility configuration information in text form into numerical representations, i.e., text feature information. This is to enable the target model to understand and use the accessibility configuration information, that is, to guide the model to understand the requirements of the multimedia files to be generated.

[0089] In some embodiments of this application, the text encoder described above may include, but is not limited to, any of the following: a word embedding-based encoder, a recurrent neural network (RNN)-based encoder, a CNN-based encoder, or a cross-modal text encoder.

[0090] It should be noted that for the specific method of how electronic devices encode accessibility configuration information through a text encoder to obtain the aforementioned text feature information, please refer to the encoding methods in related technologies. The embodiments of this application will not be described in detail here.

[0091] Step 402: The electronic device encodes the first multimedia file using a multimedia encoder to obtain multimedia file encoding information.

[0092] It is understandable that electronic devices can use a multimedia encoder to convert a first multimedia file into a numerical representation, namely the aforementioned multimedia file encoding information. This is to enable the target model to understand and use the first multimedia file.

[0093] In some embodiments of this application, when the types of the first multimedia files are different, the electronic device may use an encoder that matches the type of the first multimedia file to encode the first multimedia file.

[0094] For example, when the first multimedia file is an image, the electronic device can encode the first multimedia file using an image encoder to obtain image encoding information. The image encoder may include, but is not limited to, any of the following: a Vision Transformer (ViT) or a Normalizer-Free Net (NFNet).

[0095] For example, when the first multimedia file is a video, the electronic device can encode the first multimedia file using a video encoder to obtain video encoding information. The video encoder may include, but is not limited to, a neural audio codec.

[0096] For example, when the first multimedia file is audio, the electronic device can encode the first multimedia file using an audio encoder to obtain audio encoding information. The video encoder can be, but is not limited to, any of the following: a Transformer-based model, a deep learning-based hybrid model, or a multimodal unified paradigm.

[0097] Step 202a1: The electronic device inputs the text feature information and multimedia file encoding information into the target model.

[0098] For example, such as Figure 3A As shown, when an electronic device acquires a first multimedia file, such as an image A containing traffic lights, the electronic device can encode the image A using an image encoder to obtain the image encoding information of image A. Furthermore, electronic devices can encode the acquired accessibility configuration information using a text encoder to obtain the text feature information corresponding to the accessibility configuration information. Furthermore, the electronic device can encode the image information of image A. Text feature information corresponding to accessibility configuration information Input into the target model to perform subsequent steps.

[0099] In this embodiment, the electronic device can perform encoding operations on the accessibility configuration information and the first multimedia file respectively to obtain encoded data. Then, the target model can directly use the encoded data for processing. That is, the electronic device can provide the target model with a structured, information-intensive and computationally friendly input representation, thereby improving the convenience of subsequent model execution operations.

[0100] In some embodiments of this application, step 201a can be implemented by steps 501 and 502 as described below.

[0101] Step 501: During the process of adding noise to the first multimedia file through the target model, the electronic device performs content recognition and intent analysis on the first multimedia file to obtain recognition results and analysis results.

[0102] In some embodiments of this application, the above-mentioned "noise-adding processing of the first multimedia file" can be understood as: the electronic device gradually converts the first multimedia file into pure noise through the target model in order to obtain the above-mentioned first feature information.

[0103] In this embodiment of the application, the above-mentioned identification results may include, but are not limited to, at least one of the following: object information, scene information, and relationships between objects.

[0104] In some embodiments of this application, the aforementioned object information can be understood as information about the main objects contained in the first multimedia file. For example, objects may include, but are not limited to, at least one of the following: people, animals, objects, text information, and scene elements.

[0105] In some embodiments of this application, the electronic device can extract text information contained in a first multimedia file using Optical Character Recognition (OCR) technology.

[0106] In some embodiments of this application, the aforementioned scene information can be understood as the scene type and overall atmosphere of the scene corresponding to the first multimedia file. For example, the scene type may include, but is not limited to, any of the following: indoor, outdoor, meeting, sports, natural landscape.

[0107] In some embodiments of this application, the above-mentioned inter-object relationships may include at least one of the following relationships among the spatial relationships, interactive relationships, and causal relationships among the multiple objects contained in the first multimedia.

[0108] In this embodiment of the application, the above analysis results may include, but are not limited to, at least one of the following: emotional intent, semantic information of objects contained in the first multimedia file.

[0109] In some embodiments of this application, the aforementioned emotional intent can be understood as the emotional tone expressed by the first multimedia file, or it may simultaneously include the potential intent contained based on the emotional tone.

[0110] In some embodiments of this application, the aforementioned emotional tone may include, but is not limited to, at least one of the following: joy, sadness, tension, and anger.

[0111] In some embodiments of this application, the semantic information of the aforementioned object can be understood as the meaning corresponding to that object. For example, the first multimedia file contains abstract information such as identification charts, symbols, and metaphors, as well as the meaning corresponding to that abstract information.

[0112] Step 502: The electronic device generates first feature information based on the recognition and analysis results.

[0113] It should be noted that the specific steps of the above content recognition and intent analysis can be found in the methods of related technologies, and will not be repeated here in the embodiments of this application.

[0114] In this embodiment, the electronic device can obtain first feature information based on the recognition and analysis results corresponding to the first multimedia file. That is, the electronic device can analyze or understand the first multimedia file more accurately, so that the second multimedia file generated by the electronic device can more accurately present the content contained in the first multimedia file, reduce or avoid information presentation deviations, and improve the accuracy of the electronic device in processing files.

[0115] In some embodiments of this application, when an electronic device inputs a first multimedia file into a target model, the electronic device may first preprocess the first multimedia file and then add noise to the preprocessed data.

[0116] For example, combined Figure 3A In electronic devices, the image encoding information of image A is... When input to a target model, electronic devices can use pre-trained VAEs to map high-resolution images to a low-dimensional latent space to obtain, for example, Figure 3B The image features z are shown. Then, the target model progressively adds noise to the image features z, such as progressively obtaining the noisy image at the first time step. Noisy image at the second time step Noise image at time step T, ... , where T is the number of time steps in the noise addition process. When the number of time steps T is large enough, the image after noise addition is approximately pure noise.

[0117] In this embodiment, the electronic device can train a diffusion model in the latent space. Compared to direct diffusion in pixel space, modeling in the latent space significantly reduces data dimensionality, lowers training and sampling costs, while preserving detailed information and significantly improving visual quality.

[0118] Step 202b: The electronic device performs noise reduction processing on the first feature information based on the accessibility configuration information to obtain the second feature information.

[0119] In some embodiments of this application, step 202b can be specifically implemented by the following steps 202b1 to 202b3.

[0120] Step 202b1: The electronic device performs feature extraction processing based on the first feature information through the encoder in the image segmentation network structure of the target model to obtain the third feature information.

[0121] Step 202b2: The electronic device uses the attention module of the image segmentation network structure to perform noise reduction on the third feature information based on the accessibility configuration information to obtain the fourth feature information.

[0122] Step 202b3: The electronic device performs feature reconstruction processing based on the fourth feature information through the decoder in the image segmentation network structure to obtain the second feature information.

[0123] In some embodiments of this application, the image segmentation network structure described above can be a U-Net structure.

[0124] It is understandable that, since the core design of the above U-Net structure is a symmetrical encoder-decoder structure, the encoder in the above image segmentation network structure can be the encoder included in the U-Net structure, and the decoder in the above image segmentation network structure can be the decoder included in the U-Net structure.

[0125] It is understandable that electronic devices can combine accessibility configuration information related to user feedback needs with the process of reconstructing multimedia files based on feature information to guide the overall reconstruction process, so that the second multimedia file ultimately generated and output by the electronic device can meet the user's feedback needs.

[0126] For example, combined Figure 3B ,like Figure 3C As shown, the electronic device acquires the first feature information. In this case, the electronic device can use the encoder of the U-shaped denoising network to process the first feature information. Downsampling is performed to obtain the high-level semantic features contained in the first multimedia file. Then, the electronic device can train the image control module and the visual context attention module (QKV), i.e., the aforementioned attention module, to combine textual feature information. In this case, denoising processing is performed on the feature information, and after denoising, upsampling processing is performed through the encoder of the U-shaped denoising network to obtain the denoised feature information. Furthermore, electronic devices can process feature information. Perform similar operations to reduce noise and obtain feature information. That is, the second feature information mentioned above.

[0127] It should be noted that the number of U-shaped noise reduction networks used by the electronic device in the above noise reduction process can be determined according to actual needs, and this application embodiment does not limit it here.

[0128] In this embodiment, during the reconstruction of a multimedia file based on feature information, the electronic device can use an attention module combined with accessibility configuration information to guide the reconstruction process in accordance with user preferences or needs. Thus, the multimedia file generated and output by the electronic device can meet the user's feedback requirements. This improves the file processing efficiency of the electronic device.

[0129] Step 202c: The electronic device generates a second multimedia file based on the second feature information through the decoder of the target model.

[0130] In some embodiments of this application, the output format of the second multimedia file can be determined by the accessibility configuration information.

[0131] In some embodiments of this application, since the second multimedia file is generated based on the second feature information, and the second feature information is essentially the feature information corresponding to the first multimedia file, the second multimedia file can be understood as a multimedia file reconstructed based on the first multimedia file.

[0132] For example, combined Figure 3C ,like Figure 4 As shown, the electronic device obtains the aforementioned second feature information. In this case, the electronic device can use a decoder to obtain the second feature information. Decoding is performed to generate and output the aforementioned second multimedia file.

[0133] In some embodiments of this application, the second multimedia file may be of the same type as the first multimedia file, or it may be of a different type.

[0134] In some embodiments of this application, when the second multimedia file is of the same type as the first multimedia file, the second multimedia file may be the same as the first multimedia file or may be similar to the first multimedia file.

[0135] Step 202d: The electronic device outputs a second multimedia file through the target model.

[0136] In some embodiments of this application, the output format of the second multimedia file may include, but is not limited to, at least one of the following: subtitles, audio, images, tactile signals, and text.

[0137] In this embodiment of the application, the second multimedia file can be used to output the content of the first multimedia file in at least one way.

[0138] In some embodiments of this application, at least one of the above-described methods can be implemented by at least one of the following steps A1 to A3.

[0139] Step A1: The electronic device displays the content information of the first multimedia file in the form of subtitles.

[0140] In some embodiments of this application, when the modal combination included in the accessibility configuration information includes subtitles, that is, when the user needs the electronic device to display the content of the multimedia file in the form of subtitles, the electronic device can display the content information of the first multimedia file in the form of subtitles.

[0141] In some embodiments of this application, the display parameters of the above-mentioned subtitles can be actively set by the user or can be set by default by the electronic device.

[0142] In some embodiments of this application, when the second multimedia file is an image or video, the electronic device can automatically determine the display parameters of the subtitles based on the content of the image or video.

[0143] In some embodiments of this application, the above display parameters may include, but are not limited to, at least one of the following: display position, display font size, display color, display format, and display orientation.

[0144] In some embodiments of this application, the above-mentioned display position can be a blank space in the screen, or it can be a specified position in the screen, such as the top area, bottom area or side area.

[0145] In some embodiments of this application, the above-mentioned display format can be a static display or a dynamic display. Dynamic display can be a moving display similar to a bullet screen, or it can be a display where parameters such as display color and font size are dynamically adjusted at a fixed position, or any other feasible dynamic display.

[0146] In some embodiments of this application, the above display direction may include, but is not limited to, any of the following: from left to right, from right to left, or from top to bottom.

[0147] For example, combined Figure 4 ,like Figure 5 As shown, when the first multimedia file is image A in the example above, the electronic device can display subtitles such as "Red light on" in the blank area of ​​image A.

[0148] Step A2: The electronic device plays the content information of the first multimedia file in audio format.

[0149] In some embodiments of this application, when the modal combination included in the accessibility configuration information includes audio, that is, when the user needs the electronic device to display the content of the multimedia file in the form of audio, the electronic device can play the content information of the first multimedia file in the form of audio.

[0150] In some embodiments of this application, the playback parameters of the audio can be set by the user or by the default settings of the electronic device.

[0151] In some embodiments of this application, the playback parameters may include, but are not limited to, at least one of the following: playback path and volume.

[0152] In some embodiments of this application, the playback method described above may include, but is not limited to, any of the following: speaker, headphones, or audio equipment.

[0153] In some embodiments of this application, the audio played by the electronic device may include, but is not limited to, at least one of the following: narration, simulated ambient sound, soundscape, and emotional background music.

[0154] In some embodiments of this application, the above-mentioned narration can be a rich, accurate, and structured natural language description, which can replace visual information through language description.

[0155] In some embodiments of this application, the narration may include, but is not limited to, at least one of the following: key objects, spatial layout, object actions, events, emotions, and text content.

[0156] It should be noted that the level of description in the above narration can be set by the user. For example, the user can set the level of description to general or detailed.

[0157] In some embodiments of this application, the user can actively select the narration voice. For example, the narration voice may include, but is not limited to, any of the following: mechanical voice, child's voice, male voice, or female voice.

[0158] For example, such as Figure 5 As shown, when the first multimedia file is image A in the example above, the electronic device can play narration such as "Red light on".

[0159] In some embodiments of this application, the electronic device may provide at least one of the following: simulated ambient sound, soundscape, and emotional background music, based on the scene type corresponding to the first multimedia file.

[0160] For example, if the first multimedia file is an image or video of the ocean, the electronic device can play the sound of waves or the sound of seagulls to provide the user with simulated ambient sound corresponding to the ocean scene, thereby helping the user to understand the scene corresponding to the first multimedia file.

[0161] Step A3: The electronic device displays the content information of the first multimedia file in the form of an image.

[0162] In some embodiments of this application, when the modal combination included in the accessibility configuration information includes images, that is, when the user needs the electronic device to display the content of the multimedia file in the form of images, the electronic device can display the content information of the first multimedia file in the form of images.

[0163] In some embodiments of this application, the display parameters of the above-mentioned images can be actively set by the user or can be set by default by the electronic device.

[0164] In some embodiments of this application, when the second multimedia file is an image or video, the electronic device can automatically adjust the color parameters of the image or video content according to the content of the image or video, so that the adjusted color parameters can better meet the user's personal needs.

[0165] In some embodiments of this application, the adjustment method of the above-mentioned color parameters can be determined based on the type of color vision impairment included in the accessibility configuration information.

[0166] In some embodiments of this application, the electronic device can first determine the user's color vision impairment type, and then, based on the color confusion information corresponding to the color vision impairment type, replace the colors that the user is easily confused with, so that the user can distinguish all the colors in the picture.

[0167] In some embodiments of this application, the aforementioned color confusion information can be represented by confusion lines in the CIE chromaticity diagram.

[0168] For example, in the case where the user's color vision deficiency type is red-blindness, such as Figure 6A As shown, since users cannot accurately distinguish all color points on the same confusion line, for example, "yellow-green" with a wavelength of about 540nm and "red" with a wavelength of about 660nm are the same or similar for red-blind users, when an image contains both "yellow-green" with a wavelength of about 540nm and "red" with a wavelength of about 660nm, the electronic device can adjust the "red" with a wavelength of about 660nm to a different wavelength of "red" to facilitate color differentiation for the user.

[0169] Similarly, in cases where the user's color vision deficiency is green-blindness, such as Figure 6BAs shown, since users cannot accurately distinguish all color points on the same confusion line, for example, "blue-green" with a wavelength of about 500nm and "red" with a wavelength of about 660nm are the same or similar to green-blind users, when an image contains both "blue-green" with a wavelength of about 500nm and "red" with a wavelength of about 660nm, the electronic device can adjust the "red" with a wavelength of about 660nm to a different wavelength of "red" to facilitate color differentiation for the user.

[0170] In this way, electronic devices can reconstruct the first multimedia file by adjusting the image colors, so that users with color vision deficiencies can also distinguish different colors in the image, improving the convenience of users viewing images or videos.

[0171] In some embodiments of this application, when an electronic device outputs a second multimedia file in image form, the electronic device can perform image processing operations on the image to be output, and then output the processed image to highlight the main information of the image. For example, the electronic device can remove redundant parts of the image, such as passersby in the background.

[0172] In this embodiment, the electronic device can output a second multimedia file in different forms to provide users with multimedia information feedback in a form that meets their needs. In other words, users can conveniently and accurately obtain the information content contained in the first multimedia file in their preferred form. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device.

[0173] In some embodiments of this application, in the event of a sudden change in the content of the first multimedia file, the electronic device can promptly alert the user through tactile signals, such as vibration.

[0174] In some embodiments of this application, the electronic device can display text on its screen while outputting a second multimedia file, according to the user's needs, so as to visually present the content information of the first multimedia file to the user.

[0175] It should be noted that the above subtitles can be understood as a simplified version of the prompt information overlaid on the image or video, while the above text can be understood as a relatively complete prompt information displayed in areas other than the display area of ​​the image or video.

[0176] In some embodiments of this application, when a second multimedia file corresponds to multiple output formats, the electronic device can synchronize the accessibility content corresponding to the multiple output formats in time and adapt the spatial layout according to a strategy, so that users can simultaneously obtain the accessibility content of the multiple output formats.

[0177] In some embodiments of this application, the electronic device can output simultaneously through devices such as a screen, speaker, and tactile device, or the electronic device can also output synchronously through other connected devices to collaboratively present multimodal content reconstructed based on the first multimedia file, i.e., the aforementioned second multimedia file.

[0178] The other devices connected above may include, but are not limited to, at least one of the following: headphones, Braille display, screen reader.

[0179] It should be noted that the above connection can be a wired connection or a wireless connection, and this application embodiment does not limit it.

[0180] In this embodiment, when a user needs to obtain the information contained in the first multimedia file, the electronic device can process the first multimedia file through a target diffusion model so that the model can know the information content contained in the first multimedia file. Then, during the reconstruction of the first multimedia file based on this information content, the electronic device can combine accessibility configuration information related to the user's feedback needs, so that the second multimedia file finally generated and output by the electronic device can meet the user's feedback needs. In other words, the electronic device can display the information content contained in the first multimedia file to the user in the form of user preferences or needs. Thus, the user can obtain the information of the first multimedia file more conveniently and accurately. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device.

[0181] In the file processing method provided in this application embodiment, when a user needs to obtain the information contained in a first multimedia file, the electronic device can obtain a second multimedia file based on the first multimedia file and accessibility configuration information. That is, the electronic device can reconstruct the first multimedia file by combining the accessibility configuration information so that the final second multimedia file can meet the user's feedback needs. In other words, the electronic device can display the information content contained in the first multimedia file to the user in the form of user preferences or needs. Thus, the user can obtain the information of the first multimedia file more conveniently and accurately. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device.

[0182] In some embodiments of this application, combined with Figure 1 ,like Figure 7 As shown, prior to step 201 above, the document processing method provided in this application embodiment may further include steps 301 to 303 below.

[0183] Step 301: The electronic device displays the accessibility configuration settings interface.

[0184] In this embodiment of the application, the accessibility configuration settings interface may include at least one accessibility settings option.

[0185] In some embodiments of this application, the at least one accessibility configuration option described above may include, but is not limited to, at least one of the following: "barrier type" setting option, "user preference" setting option, "audio output" setting option, "network preference" setting option, "audio preference" setting option, and "subtitle" setting option.

[0186] In some embodiments of this application, the above-mentioned "obstacle type" setting option may include, but is not limited to, at least one of the following: red amblyopia, green amblyopia, and blue amblyopia.

[0187] In some embodiments of this application, the above-mentioned "user preference" setting option can be used to select the output format and output style, wherein the output format may include, but is not limited to, at least one of the following: image, subtitle, audio, text, haptic signal; the output style may include, but is not limited to, at least one of the following: detailed, concise.

[0188] In some embodiments of this application, the above-mentioned "audio output" setting option can be used to set the audio playback path, such as speakers, headphones, etc.; when the user selects multiple playback paths, the user can set the priority of the multiple paths.

[0189] In some embodiments of this application, the aforementioned "network preference" setting can be used to configure the network settings for an electronic device to reconstruct a second multimedia file based on a first multimedia file. Examples include Wi-Fi and data networks.

[0190] Specifically, if the "Network Preferences" settings include WIFI, it means that when the electronic device is connected to WIFI, the user can trigger the electronic device to execute the scheme of this application; if the "Network Preferences" settings include data network, it means that when the electronic device is using data network, the user can trigger the electronic device to execute the scheme of this application.

[0191] In some embodiments of this application, the above-mentioned "audio preference" setting option can be used to set the type of audio to be played, such as narration or simulated ambient sound effects.

[0192] In some embodiments of this application, the above-mentioned "subtitle" setting option can be used to set whether to display subtitles. For example, "on" or "off".

[0193] Step 302: The electronic device receives the user's first input for at least one accessibility setting option.

[0194] In some embodiments of this application, the first input described above can be used to edit or adjust the content of at least a portion of the accessibility settings options.

[0195] In some embodiments of this application, the first input mentioned above may include, but is not limited to, any of the following: click input, voice input, gesture input, or other feasible inputs. This application does not limit this.

[0196] For example, a mobile phone is used as an example electronic device. The mobile phone can receive the user's first input on at least one accessibility setting option of the accessibility configuration settings interface, such as... Figure 8 As shown, the edited accessibility configuration settings interface 20 is displayed, and this interface 20 can include six accessibility settings options. These include "Disability Type" settings, "User Preferences" settings, "Audio Output" settings, "Network Preferences" settings, "Audio Preferences" settings, and "Subtitles" settings. Specifically, the "Disability Type" setting is set to "Red for Visually Impaired"; the "User Preferences" setting is set to "Subtitles / Audio / Simplified," meaning the user wants the phone to display subtitles for multimedia files and play the corresponding audio, while preferring a simplified style; the "Audio Output" setting is set to "Headphones / Speaker," meaning that during the output of a second multimedia file, the phone can prioritize detecting whether headphones are connected, playing through headphones if connected, and through the speaker if not; the "Network Preferences" setting is set to "Wi-Fi / Data Network"; the "Audio Preferences" setting is set to "Narration / Simulated Ambient Sound," meaning the phone can play simulated ambient sound while playing narration to create an immersive auditory experience for the user; and the "Subtitles" setting is set to "On."

[0197] Meanwhile, the accessibility configuration settings interface 20 also includes an exit control, a save control, and a modify control.

[0198] Step 303: The electronic device responds to the first input and determines the accessibility configuration information.

[0199] In some embodiments of this application, the accessibility configuration information described above may be text information that is a combination or summary of information from an electronic device based on at least one accessibility setting option.

[0200] In this embodiment of the application, the accessibility configuration information may include, but is not limited to, at least one of the following: user disability type information and user requirement information.

[0201] In this embodiment of the application, the above-mentioned user requirement information can be used to characterize the user's requirements for the output method of multimedia files.

[0202] For example, combined Figure 8 According to information from at least one accessibility setting option, the user's disability type information can be "red visual impairment", and the user's needs information can be to output a second multimedia file in a "brief" form through "subtitles" and "audio". The "audio" can be output using "headphones" and "speakers", and the output audio content is "narration" and "simulated ambient sound". At the same time, the second multimedia file can be reconstructed based on the first multimedia file when the mobile phone is using "WIFI" or "data network".

[0203] In this embodiment, the electronic device can obtain accessibility configuration information based on the user's settings. Since the electronic device uses this accessibility configuration information when obtaining the second multimedia file based on the first multimedia file, the second multimedia information can meet the user's feedback needs. Furthermore, the user can obtain the information of the first multimedia file more conveniently and accurately, thus improving the file processing effect of the electronic device.

[0204] In some embodiments of this application, the file processing method provided in this application may further include the following steps 601 and 602.

[0205] Step 601: The electronic device obtains user feedback information.

[0206] Step 602: The electronic device adjusts the model parameters of the target model based on user feedback.

[0207] In this embodiment of the application, the aforementioned user feedback information can be used to characterize the user's feedback on the use of the second multimedia file.

[0208] In some embodiments of this application, the aforementioned user feedback information may include at least one of the following: implicit feedback and explicit feedback.

[0209] In some embodiments of this application, the aforementioned implicit feedback may include, but is not limited to, at least one of the following: the user skips a certain description, repeatedly requests certain information, or adjusts settings.

[0210] In some embodiments of this application, the above-mentioned display feedback may include, but is not limited to, at least one of the following: scoring, error correction operation.

[0211] In some embodiments of this application, the electronic device can acquire embedded data to obtain the aforementioned user feedback information.

[0212] It should be noted that the specific methods for electronic devices to acquire embedded data can be found in related technologies, and will not be elaborated here in the embodiments of this application.

[0213] In this embodiment, the electronic device can use user feedback information to fine-tune or incrementally learn the target model, thereby continuously improving the accuracy, naturalness, personalization, and user satisfaction of the generated content, so as to provide users with a second multimedia file that better meets their needs.

[0214] To illustrate the various scenarios in which the embodiments of this application can be applied, and in conjunction with the various implementation schemes of the embodiments of this application described above, specific examples are given below to explain the implementation process of the embodiments of this application in various scenarios. A mobile phone is used as an example for illustration.

[0215] When a user with color vision impairment needs to view traffic lights, they can use their phone's camera app to photograph the traffic lights, allowing the phone to capture a real-time video recording of the traffic lights. Simultaneously, the phone can access accessibility settings configured by the user, such as displaying subtitles and playing audio, with the output style set to "simplified".

[0216] For a specific image frame in the real-time recorded video, such as preview image B, the mobile phone can encode preview image B using an image encoder to obtain the image encoding information of preview image B. Furthermore, the mobile phone can encode the accessibility configuration information using a text encoder to obtain the text feature information corresponding to the accessibility configuration information. Furthermore, the mobile phone can transmit the image encoding information of the captured preview image B. Text feature information corresponding to accessibility configuration information Input into the target model.

[0217] Therefore, the mobile phone can use a pre-trained VAE to map a high-resolution image to a low-dimensional latent space to obtain image features z. Then, the target model progressively adds noise to the image features z, such as progressively obtaining a noisy image at the first time step. Noisy image at the second time step Noise image at time step T, ... Then, the mobile phone can process the noisy image through the encoder of the U-shaped denoising network. Downsampling is performed to obtain the high-level semantic features contained in the first multimedia file. Then, the mobile phone can train the image control module and the visual context attention module (QKV) to combine text feature information. The feature information is denoised, and then upsampling is performed through the encoder of the U-shaped denoising network to obtain the denoised feature information. Furthermore, mobile phones can analyze feature information. Perform similar operations to reduce noise and obtain feature information. This refers to the second feature information. Then, the second feature information is obtained on the mobile phone. In this case, the mobile phone can use a decoder to analyze the second feature information. After decoding, the phone can display the captured preview image B, and show brief subtitles such as "Red light on" in the blank area of ​​the captured preview image B, along with a brief narration such as "Red light on." Thus, the user can know the color of the traffic light.

[0218] In other words, during the reconstruction of the captured preview image B, the electronic device can incorporate accessibility configuration information related to user feedback needs, so that the subtitles and narration generated and output by the electronic device can meet the user's feedback needs. That is to say, the electronic device can display the information content contained in the captured preview image B to the user in the form of user preferences or needs. Thus, the user can obtain the information of the captured preview image B more conveniently and accurately, thereby improving the file processing effect of the electronic device.

[0219] It should be noted that the above-described method embodiments, or the various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0220] It should be noted that the file processing method provided in this application embodiment can be executed by a file processing device. This application embodiment uses a file processing device executing the file processing method as an example to illustrate the file processing device provided in this application embodiment.

[0221] Figure 9 A schematic diagram of a possible structure of the document processing apparatus involved in an embodiment of this application is shown. For example... Figure 9 As shown, the file processing device 70 may include: an acquisition module 71 and a processing module 72;

[0222] The acquisition module 71 is used to acquire a first multimedia file and accessibility configuration information. The accessibility configuration information includes at least one of the following: user disability type information and user demand information. The user demand information is used to characterize the user's demand for the output method of the multimedia file.

[0223] The processing module 72 is used to obtain a second multimedia file based on the first multimedia file and accessibility configuration information obtained by the acquisition module 71. The second multimedia file is used to output the content of the first multimedia file in at least one way.

[0224] In one possible implementation, the file processing apparatus 70 provided in this application embodiment may further include: a display module, a receiving module, and a determining module; the display module is configured to display an accessibility configuration settings interface before the acquisition module acquires the first multimedia file and accessibility configuration information, the accessibility configuration settings interface including at least one accessibility setting option; the receiving module is configured to receive a first input from a user to at least one accessibility setting option displayed by the display module; the determining module is configured to determine the accessibility configuration information in response to the first input received by the receiving module.

[0225] In one possible implementation, the processing module 72 is specifically configured to: input the first multimedia file and accessibility configuration information into the target model to obtain first feature information, the first feature information including description information and key features of the first multimedia file; and perform noise reduction processing on the first feature information based on the accessibility configuration information to obtain second feature information; and generate a second multimedia file based on the second feature information through the decoder of the target model; and output the second multimedia file through the target model.

[0226] In one possible implementation, the processing module 72 is further configured to encode the accessibility configuration information using a text encoder to obtain text feature information before inputting the first multimedia file and accessibility configuration information into the target model; and to encode the first multimedia file using a multimedia encoder to obtain multimedia file encoding information; specifically, the processing module 72 is configured to input the text feature information and multimedia file encoding information into the target model.

[0227] In one possible implementation, the processing module 72 is specifically used to perform content recognition and intent analysis on the first multimedia file during the process of adding noise to the first multimedia file through the target model, and obtain recognition results and analysis results. The recognition results include at least one of the following: object information, scene information, and relationship between objects. The analysis results include at least one of the following: emotional intent and semantic information of the objects contained in the first multimedia file. Based on the recognition results and analysis results, first feature information is generated.

[0228] In one possible implementation, the processing module 72 is specifically used to perform feature extraction processing based on the first feature information through the encoder in the image segmentation network structure of the target model to obtain the third feature information; and to perform noise reduction processing on the third feature information based on the accessibility configuration information through the attention module in the image segmentation network structure to obtain the fourth feature information; and to perform feature reconstruction processing based on the fourth feature information through the decoder in the image segmentation network structure to obtain the second feature information.

[0229] In one possible implementation, the processing module 72 is specifically used to: display the content information of the first multimedia file in the form of subtitles; or, play the content information of the first multimedia file in the form of audio; or, display the content information of the first multimedia file in the form of images.

[0230] In one possible implementation, the acquisition module 71 is further configured to acquire user feedback information, which is used to characterize the user's feedback on the use of the second multimedia file; the processing module 72 is further configured to adjust the model parameters of the target model based on the user feedback information.

[0231] In the file processing apparatus provided in this application embodiment, when a user needs to obtain the information contained in a first multimedia file, the file processing apparatus can obtain a second multimedia file based on the first multimedia file and accessibility configuration information. That is, the file processing apparatus can reconstruct the first multimedia file by combining the accessibility configuration information so that the final second multimedia file can meet the user's feedback needs. In other words, the file processing apparatus can display the information content contained in the first multimedia file to the user in the form of user preferences or needs. Thus, the user can obtain the information of the first multimedia file more conveniently and accurately. This improves the diversity of the file processing apparatus's processing methods for multimedia files and enhances the file processing effect of the file processing apparatus.

[0232] The file processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0233] The file processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0234] The file processing apparatus provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0235] Optionally, such as Figure 10 As shown, this application embodiment also provides an electronic device 90, including a processor 91 and a memory 92. The memory 92 stores a program or instructions that can run on the processor 91. When the program or instructions are executed by the processor 91, they implement the various steps of the above-described file processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0236] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0237] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0238] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0239] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0240] The processor 110 is configured to acquire a first multimedia file and accessibility configuration information, the accessibility configuration information including at least one of the following: user disability type information and user demand information, the user demand information being used to characterize the user's demand for the output method of the multimedia file; and based on the first multimedia file and the accessibility configuration information, to obtain a second multimedia file, the second multimedia file being used to output the content of the first multimedia file in at least one way.

[0241] Optionally, the display unit 106 is configured to display an accessibility configuration settings interface before acquiring the first multimedia file and accessibility configuration information, the accessibility configuration settings interface including at least one accessibility setting option; the user input unit 107 is configured to receive a first input from a user for at least one accessibility setting option; and the processor 110 is further configured to determine the accessibility configuration information in response to the first input.

[0242] Optionally, the processor 110 is specifically configured to input the first multimedia file and accessibility configuration information into the target model to obtain first feature information, the first feature information including description information and key features of the first multimedia file; and perform noise reduction processing on the first feature information based on the accessibility configuration information to obtain second feature information; and generate a second multimedia file based on the second feature information through the decoder of the target model; and output the second multimedia file through the target model.

[0243] Optionally, the processor 110 is further configured to encode the accessibility configuration information using a text encoder to obtain text feature information before inputting the first multimedia file and accessibility configuration information into the target model; and to encode the first multimedia file using a multimedia encoder to obtain multimedia file encoding information; specifically, the processor 110 is configured to input the text feature information and multimedia file encoding information into the target model.

[0244] Optionally, the processor 110 is specifically configured to perform content recognition and intent analysis on the first multimedia file during the process of adding noise to the first multimedia file through the target model, and obtain recognition results and analysis results. The recognition results include at least one of the following: object information, scene information, and relationship between objects. The analysis results include at least one of the following: emotional intent and semantic information of the objects contained in the first multimedia file. Based on the recognition results and analysis results, the processor generates first feature information.

[0245] Optionally, the processor 110 is specifically configured to perform feature extraction processing based on the first feature information through the encoder in the image segmentation network structure of the target model to obtain the third feature information; and to perform noise reduction processing on the third feature information based on the accessibility configuration information through the attention module in the image segmentation network structure to obtain the fourth feature information; and to perform feature reconstruction processing based on the fourth feature information through the decoder in the image segmentation network structure to obtain the second feature information.

[0246] Optionally, the processor 110 is specifically configured to display the content information of the first multimedia file in the form of subtitles; or, play the content information of the first multimedia file in the form of audio; or, display the content information of the first multimedia file in the form of images.

[0247] Optionally, the processor 110 is also configured to acquire user feedback information, which characterizes the user's feedback on the use of the second multimedia file; and adjust the model parameters of the target model based on the user feedback information.

[0248] In the electronic device provided in this application embodiment, when a user needs to obtain the information contained in a first multimedia file, the electronic device can obtain a second multimedia file based on the first multimedia file and accessibility configuration information. That is, the electronic device can reconstruct the first multimedia file by combining the accessibility configuration information so that the final second multimedia file can meet the user's feedback needs. In other words, the electronic device can display the information content contained in the first multimedia file to the user in the form of user preferences or needs. Thus, the user can obtain the information of the first multimedia file more conveniently and accurately. This improves the diversity of the electronic device's processing methods for multimedia files and enhances the file processing effect of the electronic device.

[0249] The electronic device provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0250] For details on the beneficial effects of the various implementation methods in this embodiment, please refer to the beneficial effects of the corresponding implementation methods in the above method embodiments. To avoid repetition, these will not be repeated here.

[0251] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0252] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0253] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0254] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0255] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0256] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0257] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0258] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0259] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0260] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0261] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A file processing method, characterized in that, The method includes: Obtain a first multimedia file and accessibility configuration information, wherein the accessibility configuration information includes at least one of the following: user disability type information and user demand information, wherein the user demand information is used to characterize the user's demand for the output method of the multimedia file; Based on the first multimedia file and the accessibility configuration information, a second multimedia file is obtained, and the second multimedia file is used to output the content of the first multimedia file in at least one way.

2. The method according to claim 1, characterized in that, Before obtaining the first multimedia file and accessibility configuration information, the method further includes: Display an accessibility configuration settings interface, which includes at least one accessibility settings option; Receive first input from the user regarding the at least one accessibility setting option; In response to the first input, the accessibility configuration information is determined.

3. The method according to claim 1, characterized in that, The process of obtaining a second multimedia file based on the first multimedia file and the accessibility configuration information includes: The first multimedia file and the accessibility configuration information are input into the target model to obtain the first feature information, which includes the description information and key features of the first multimedia file. Based on the accessibility configuration information, the first feature information is denoised to obtain the second feature information; The second multimedia file is generated based on the second feature information using the decoder of the target model. The second multimedia file is output through the target model.

4. The method according to claim 3, characterized in that, Before inputting the first multimedia file and the accessibility configuration information into the target model, the method further includes: The accessibility configuration information is encoded using a text encoder to obtain text feature information; The first multimedia file is encoded using a multimedia encoder to obtain multimedia file encoding information; The step of inputting the first multimedia file and the accessibility configuration information into the target model includes: The text feature information and the multimedia file encoding information are input into the target model.

5. The method according to claim 3 or 4, characterized in that, The step of inputting the first multimedia file and the accessibility configuration information into the target model to obtain the first feature information includes: During the process of adding noise to the first multimedia file through the target model, content recognition and intent analysis are performed on the first multimedia file to obtain recognition results and analysis results. The recognition results include at least one of the following: object information, scene information, and relationships between objects. The analysis results include at least one of the following: emotional intent and semantic information of the objects contained in the first multimedia file. Based on the recognition results and the analysis results, the first feature information is generated.

6. The method according to claim 3 or 4, characterized in that, The step of denoising the first feature information based on the accessibility configuration information to obtain the second feature information includes: The encoder in the image segmentation network structure of the target model performs feature extraction processing based on the first feature information to obtain the third feature information; The attention module of the image segmentation network structure performs noise reduction on the third feature information based on the accessibility configuration information to obtain the fourth feature information. The second feature information is obtained by performing feature reconstruction processing based on the fourth feature information through the decoder in the image segmentation network structure.

7. The method according to claim 1, characterized in that, The process of obtaining the second multimedia file includes: The content information of the first multimedia file is displayed as subtitles; or, Play the content information of the first multimedia file in audio format; or... The content information of the first multimedia file is displayed in the form of an image.

8. The method according to claim 1, characterized in that, The method further includes: Obtain user feedback information, which is used to characterize the user's feedback on the use of the second multimedia file; Based on the user feedback, the model parameters of the target model are adjusted.

9. A document processing device, characterized in that, The device includes: an acquisition module and a processing module; The acquisition module is used to acquire a first multimedia file and accessibility configuration information. The accessibility configuration information includes at least one of the following: user disability type information and user demand information. The user demand information is used to characterize the user's demand for the output method of the multimedia file. The processing module is used to obtain a second multimedia file based on the first multimedia file obtained by the acquisition module and the accessibility configuration information, and the second multimedia file is used to output the content of the first multimedia file in at least one way.

10. The apparatus according to claim 9, characterized in that, The processing module is specifically used for: The first multimedia file and the accessibility configuration information are input into the target model to obtain the first feature information, which includes the description information and key features of the first multimedia file. Based on the accessibility configuration information, the first feature information is denoised to obtain the second feature information; The second multimedia file is generated based on the second feature information using the decoder of the target model. The second multimedia file is output through the target model.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the file processing method as described in any one of claims 1 to 8.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the file processing method as described in any one of claims 1 to 8.

Images