Image transmission method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an image transmission method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a target image sent by an image sender; determining a text recognition identifier of the target image based on the image type of the target image, and recognizing text information of the target image based on the text recognition identifier; sending the text information and the text recognition identifier to an image receiver, so that the image receiver determines an image display mode based on the current equipment environment, and displays the image information based on the image display mode; the image display mode comprises a first mode and a second mode, the first mode is used for displaying the text information and the text recognition identifier, and the second mode is used for displaying the text information, the text recognition identifier and the target image. By adopting the method, the instantaneity of image transmission can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to an image transmission method, apparatus, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] In current instant messaging, picture messages provide users with rich ways of expression and intuitive visual experience. However, since pictures are relatively large, they require more network bandwidth for transmission, which may result in slower transmission speeds when network conditions are poor, causing delays in users receiving and viewing pictures.

[0003] Moreover, for devices with lower configuration, insufficient storage space may result in a failure in image downloading, so that users cannot view images in time, resulting in low real-time performance of image instant messaging. Summary of the invention

[0004] Based on this, it is necessary to provide an image transmission method, device, computer equipment, computer-readable storage medium and computer program product that can improve the real-time performance of picture instant messaging in response to the above technical problems.

[0005] In a first aspect, the present application provides an image transmission method, comprising:

[0006] Get the target image sent by the image sender;

[0007] Determining a text recognition identifier of the target image based on an image type of the target image, and recognizing text information of the target image based on the text recognition identifier;

[0008] The text information and the text recognition identifier are sent to the image receiver so that the image receiver determines the image display mode based on the current device environment and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition identifier, and the second mode is used to display the text information, the text recognition identifier and the target image.

[0009] In a second aspect, the present application further provides an image transmission device, comprising:

[0010] An acquisition module, used for acquiring a target image sent by an image sender;

[0011] A text recognition module, used to determine a text recognition identifier of the target image based on the image type of the target image, and recognize text information of the target image based on the text recognition identifier;

[0012] An image sending module is used to send the text information and the text recognition identifier to an image receiver, so that the image receiver determines an image display mode based on the current device environment and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition identifier, and the second mode is used to display the text information, the text recognition identifier and the target image.

[0013] In a third aspect, the present application provides an image transmission method, comprising:

[0014] Obtaining text information and a text recognition identifier corresponding to the target image sent by the server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier;

[0015] The image display mode is determined according to the current device environment, and the image information is displayed based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition logos, and the second mode is used to display text information, text recognition logos and target images.

[0016] In a fourth aspect, the present application further provides an image transmission device, comprising:

[0017] An acquisition module is used to acquire text information and a text recognition identifier corresponding to a target image sent by a server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier;

[0018] The display module is used to determine the image display mode according to the current device environment and display image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition logo, and the second mode is used to display text information, text recognition logo and target image.

[0019] In a fifth aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0020] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0021] In a seventh aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps in the above method.

[0022] The above-mentioned image transmission method, device, computer equipment, computer-readable storage medium and computer program product, the server receives the target image sent by the image sender, determines the text recognition identifier according to the image type of the target image, and identifies the text information of the target image according to the text recognition identifier, which can ensure the recognition accuracy of the text information of target images of different image types, and realizes the extraction of image information of the target image through the text information and the text recognition identifier. Then, the server sends the text information and the text recognition identifier to the image receiver, and can transmit the image information of the target image to the image receiver in real time, thereby realizing the real-time transmission of the image information of the target image.

[0023] The image receiver receives the text information and text recognition identifier corresponding to the target image sent by the server, determines the image display mode according to the current device environment, displays the image information of the target image according to the image display mode, displays the text information and text recognition identifier in the first mode, and displays the text information, text recognition identifier and target image in the second mode. It can automatically match the corresponding display mode according to the current device environment, display the image information of the target image in time, and ensure the real-time transmission of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A diagram showing an application environment of an image transmission method in an embodiment;

[0026] Figure 2 is a schematic flow chart of an image transmission method in one embodiment;

[0027] Figure 3 is a flow chart of an image transmission method in another embodiment;

[0028] Figure 4 A schematic diagram of displaying image information in one embodiment;

[0029] Figure 5 is a schematic diagram of a process of image transmission in one embodiment;

[0030] Figure 6 is a structural block diagram of an image transmission device in one embodiment;

[0031] Figure 7 is a structural block diagram of an image transmission device in another embodiment;

[0032] Figure 8 is an internal structure diagram of a computer device in one embodiment;

[0033] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The image transmission method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the image sender 102 and the image receiver 106 communicate with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. In an exemplary embodiment, the server 104 obtains the target image sent by the image sender 102; the server 104 determines the text recognition identifier of the target image based on the image type of the target image, and identifies the text information of the target image based on the text recognition identifier; the server 104 sends the text information and the text recognition identifier to the image receiver 106, so that the image receiver 106 determines the image display mode based on the current device environment, and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition identifiers, and the second mode is used to display text information, text recognition identifiers and target images. The image sender 102 and the image receiver 106 may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0036] In an exemplary embodiment, Figure 2 As shown, a method for image transmission is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0037] Step 202: Acquire the target image sent by the image sender.

[0038] Step 204 : determining a text recognition identifier of the target image based on the image type of the target image, and recognizing text information of the target image based on the text recognition identifier.

[0039] The target image refers to the image that the image sender needs to send to the image receiver at present. The image type refers to whether the target image contains pure text. Pure text refers to an image that contains only text content. The text recognition identifier is an identifier that indicates the recognition method of the text information of the target image. The text information is information that represents the content of the target image. The information format is text format, which can be text content extracted from the image or descriptive text generated based on the image content.

[0040] Exemplarily, the server obtains an image transmission request sent by an image sender, the image transmission request includes a target image that the image sender needs to send to an image receiver, stores the target image, and identifies the image type of the target image, the image type includes a text image type and a non-text image type, the text image type refers to an image type whose image content is pure text information, and the non-text image type refers to an image type whose image content is not pure text information. The server can perform image analysis on the target image to obtain the image type of the target image, and determine a text recognition identifier corresponding to the target image according to the image type of the target image. Then, the text recognition algorithm corresponding to the text recognition identifier is called to recognize text information of the target image.

[0041] In an exemplary embodiment, the server performs image analysis on the target image to obtain the image type of the target image. The server may extract image features from the target image. The image features include text features and non-text features. Non-text features include features such as color, texture, shape, etc. in the target image. Text features include features such as characters. Then, the image type of the target image is determined based on the proportion of text features in the image features. For example, when the proportion of text features in the image features is greater than a preset proportion threshold, the image type of the target image is determined to be a text image type. Generally, the preset proportion threshold is 100%.

[0042] In one embodiment, when the image sender transmits an image to the image receiver, the server acts as a transfer party to process the image sent by the image sender and forward it to the image receiver. The server may extract image information from the image sent by the image sender and forward the image information to the image receiver; or the server may send the image information and the image to the image receiver in a preset display format.

[0043] Step 206, sending the text information and the text recognition identifier to the image receiver, so that the image receiver determines the image display mode based on the current device environment, and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition identifier, and the second mode is used to display the text information, the text recognition identifier and the target image.

[0044] The current device environment refers to the remaining environmental resources in the image receiver that can be used to receive and display images, and represents the image receiver's current ability to receive and display images. The image display mode refers to the display method of the image receiver for displaying the image information of the received image. The image information includes at least two of text information, text recognition identifier, and target image.

[0045] Exemplarily, the image transmission request sent by the image sender also includes the receiver address corresponding to the image receiving method. After obtaining the text information and text recognition identifier of the target image, the server communicates with the image receiver according to the receiver address. The image storage address of the target image in the server is obtained. The image storage address is used by the image receiver to obtain the target image from the server. The server generates an image receiving request according to the text information, text recognition identifier and storage address corresponding to the target image, and sends the image receiving request to the image receiver.

[0046] The image receiver receives the image receiving request sent by the server, and determines whether the current device environment meets the image display requirements. The image display requirements refer to the environment resources required for the target image to be successfully transmitted to the image receiver and successfully displayed by the image receiver. When the current device environment does not meet the image display requirements, the image receiver displays the text information and text recognition identifier in a first mode; when the current device environment meets the image display requirements, the image receiver displays the text information and text recognition identifier in a second mode, and obtains the target image from the server according to the image storage address for display.

[0047] In the above-mentioned image transmission method, the server receives the target image sent by the image sender, determines the text recognition identifier according to the image type of the target image, and identifies the text information of the target image according to the text recognition identifier, which can ensure the recognition accuracy of the text information of target images of different image types, and realizes the extraction of image information of the target image through the text information and the text recognition identifier. Then, the server sends the text information and the text recognition identifier to the image receiver, and can transmit the image information of the target image to the image receiver in real time, thereby realizing the real-time transmission of the image information of the target image.

[0048] The image receiver receives the text information and text recognition identifier corresponding to the target image sent by the server, determines the image display mode according to the current device environment, displays the image information of the target image according to the image display mode, displays the text information and text recognition identifier in the first mode, and displays the text information, text recognition identifier and target image in the second mode. It can automatically match the corresponding display mode according to the current device environment, display the image information of the target image in time, and ensure the real-time transmission of the target image.

[0049] In an exemplary embodiment, Figure 3 As shown, step 204, based on the image type of the target image, determines the text recognition identifier of the target image, and identifies the text information of the target image based on the text recognition identifier, including:

[0050] When the image type of the target image is a text image type, determining the text recognition identifier of the target image as a character recognition identifier, performing character recognition on the target image, and obtaining text information corresponding to the target image;

[0051] When the image type of the target image is a non-text image type, the text recognition identifier of the target image is determined to be the model recognition identifier, and the target image is input into the target image-to-text model to obtain text information corresponding to the target image.

[0052] The text recognition identifier represents a text recognition method for performing text recognition on a target image. The model recognition identifier represents a text recognition method for generating text information from a target image using a network model. The target image-to-text model is a model for generating descriptive text for the image content based on an image.

[0053] Exemplarily, the server identifies the image type of the target image. When the image type of the target image is a text image type, it indicates that the image content of the target image is plain text. The server determines that the text recognition identifier of the target image is a text recognition identifier, such as "text recognition". The server uses the text recognition algorithm corresponding to the text recognition identifier to perform character recognition on the text in the target image and extract the text information of the target image. The text information indicates the text content extracted from the target image.

[0054] When the image type of the target image is a non-text image type, it means that the image content of the target image includes non-pure text content such as graphics, and the text recognition identifier of the target image is determined to be a model recognition identifier, such as "image-to-text", and a pre-trained target image-to-text model is obtained according to the model recognition identifier. The target image is input into the target image-to-text model for text generation, and text information corresponding to the target image is obtained. The text information represents descriptive text content generated according to the image content of the target image.

[0055] In an exemplary embodiment, the server uses a text recognition algorithm corresponding to a text recognition identifier to perform character recognition on the text in the target image and extract text information of the target image. The target image may be preprocessed, such as denoising, binarization, border removal, etc.; then, character segmentation is performed on the target image, and the text in the target image is segmented into single characters or words. Feature extraction is performed on the segmented characters to obtain character features, such as the outline, shape, size, etc. of the characters; the extracted character features are matched with a predefined character library to identify the corresponding characters, and the text information corresponding to the target image is generated according to the order in which the characters in the target image are arranged according to the identified characters.

[0056] In this embodiment, by performing text recognition on a target image of a pure text image type and using a target image-to-text model to generate text information for a target image of a non-pure text image type, the recognition accuracy of the text information of the target image can be improved.

[0057] In an exemplary embodiment, the image transmission method further includes:

[0058] Obtain a training image set corresponding to the non-text image type;

[0059] Based on the image scene types corresponding to each training image in the training image set, determine the label dimension corresponding to each training image, annotate key information of each training image based on the label dimension, and obtain a training label set corresponding to the training image set;

[0060] Based on the training image set and the training label set corresponding to the training image set, the initial image-to-text model is supervised trained to obtain the target image-to-text model.

[0061] The image scene type refers to the type of situation presented in the image, such as a person image, a landscape image, etc. The label dimension refers to the label attribute that labels the specified content in the image according to different image scene types, such as the label dimensions of the person's appearance, action, expression, and background in a person image, or the label dimensions of natural landscape, weather conditions, and color matching in a landscape image.

[0062] Specifically, the server obtains a training image set corresponding to a non-text image type, determines the label dimensions corresponding to each training image according to the image scene types corresponding to each training image in the training image set, obtains the key information annotated by each training image in each label dimension, such as annotated keywords or descriptive paragraphs, and obtains the training label set corresponding to the training image set. An initial image-to-text model is obtained, and supervised training is performed on the initial image-to-text model according to the training image set and the training label set corresponding to the training image set. The training image set can be input into the initial image-to-text model for text generation, and the training text information corresponding to the training image set is obtained. The training text information is compared with the corresponding label dimension. When the matching degree between the training text information and the corresponding label dimension reaches the matching degree requirement, it is determined that the training of the initial image-to-text model is completed, and the target image-to-text model is obtained.

[0063] In an exemplary embodiment, object features are extracted from each training image in the training image set, such as table and chair, food, and clothing, and the attention level of each object feature in each training image is calculated. The attention level of each object feature can be calculated based on the degree of association between the object features, such as the greater the degree of association, the higher the attention level. According to the attention level of each object feature, foreground features and background features are determined in each object feature, and the training image is divided into foreground image and background image according to the foreground features and background features. The label dimensions corresponding to the foreground image and the background image are determined according to the image scene type of the training image, and the key information annotated in each label dimension of the foreground image and the background image in the training image is obtained, wherein the amount of key information corresponding to the foreground image is greater than the amount of key information corresponding to the background image.

[0064] In this embodiment, supervised training is performed on the initial image-to-text model using a training image set and a training label set corresponding to the training image set, thereby ensuring the accuracy of image-to-text generation by the target image-to-text model.

[0065] In an exemplary embodiment, step 202, obtaining a target image sent by an image sender, includes:

[0066] Obtaining a coded image sent by an image sender after encoding an initial image;

[0067] When the encoding format corresponding to the image sender matches the decoding format corresponding to the image receiver, the encoded image is used as the target image;

[0068] When the encoding format corresponding to the image sender does not match the decoding format of the image receiver, the encoded image is decoded to obtain an initial image;

[0069] The current encoding format is determined based on the decoding format corresponding to the image receiver, and the initial image is re-encoded according to the current encoding format to obtain the target image.

[0070] The initial image refers to the image that the image sender needs to send and has not been encoded. The current encoding format refers to the format that the server currently needs to encode when sending the image to the image receiver.

[0071] Exemplarily, the image sender determines that the image currently to be sent to the image receiver is the initial image, encodes the initial image according to the local encoding format to obtain an encoded image, generates an image transmission request according to the encoded image and the receiver address of the image receiver, and sends the image transmission request to the server.

[0072] The server determines the decoding format corresponding to the image receiver according to the receiver address. When the encoding format corresponding to the image sender matches the decoding format corresponding to the image receiver, the encoded image is used as the target image. When the encoding format corresponding to the image sender does not match the decoding format of the image receiver, the encoded image is decoded to obtain the initial image. Then the current encoding format is determined according to the decoding format corresponding to the image receiver, and the initial image is re-encoded according to the current encoding format to obtain the target image.

[0073] In this embodiment, by re-encoding the initial image according to the encoding and decoding format corresponding to the image receiver, it is possible to ensure that the image receiver can smoothly decode and display the target image, thereby improving the immediacy of image transmission.

[0074] In an exemplary embodiment, Figure 3 As shown,

[0075] Step 302, obtaining text information and a text recognition identifier corresponding to the target image sent by the server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier;

[0076] Step 304, determine the image display mode according to the current device environment, and display the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition logo, and the second mode is used to display text information, text recognition logo and target image.

[0077] Exemplarily, the image receiver receives the image receiving request sent by the server, and determines whether the current device environment meets the image display requirements. The image display requirements refer to the environment resources required for the target image to be successfully transmitted to the image receiver and successfully displayed by the image receiver. When the current device environment does not meet the image display requirements, the image receiver displays the text information and text recognition identifier in a first mode; when the current device environment meets the image display requirements, the image receiver displays the text information and text recognition identifier in a second mode, and obtains the target image from the server according to the image storage address for display.

[0078] The above-mentioned image transmission method, through the image receiver receiving the text information and text recognition identifier corresponding to the target image sent by the server, determines the image display mode according to the current device environment, displays the image information of the target image according to the image display mode, displays the text information and text recognition identifier in the first mode, and displays the text information, text recognition identifier and target image in the second mode. It can automatically match the corresponding display mode according to the current device environment, display the image information of the target image in time, and ensure the real-time transmission of the target image.

[0079] In an exemplary embodiment, step 304, determining an image display mode according to the current device environment, and displaying image information based on the image display mode, includes:

[0080] When the current device environment does not meet the image display requirements for the target image, display the text information and the text recognition mark based on the first mode;

[0081] When the current device environment meets the image display requirement for the target image, the target image is obtained from the server, and the text information, the text recognition mark and the target image are displayed based on the second mode.

[0082] Exemplarily, the image receiver determines whether the image display requirements are met based on the current device environment. When the current device environment does not meet the image display requirements for the target image, such as when the device network of the image receiver is in a weak network environment or the device resources of the image receiver are insufficient, the text information and text recognition mark of the target image are displayed according to the first mode. When the current device environment meets the image display requirements for the target image, such as when the device network environment of the image receiver is good or the device resources of the image receiver are sufficient, the text information, text recognition mark and target image are displayed according to the second mode. Among them, when the target image is an encoded image, the image receiver obtains the target image from the server according to the storage address of the target image, and decodes the target image according to the configured decoding format to obtain the initial image, and displays the text information, text recognition mark and initial image according to the preset format.

[0083] In this embodiment, by selecting corresponding display modes in different current device environments, the instantaneous display of the image information of the target image can be ensured.

[0084] In an exemplary embodiment, the image transmission method further includes:

[0085] In the first mode, an image loading control corresponding to the target image is displayed;

[0086] In response to a trigger operation on the image loading control, obtaining an update device environment;

[0087] When the device environment is updated to meet the image display requirements for the target image, the target image is obtained from the server and displayed.

[0088] The image loading control is a control provided by the image receiver to the user to select whether to display the image. The updating device environment refers to the device environment obtained by the image receiver after re-evaluating the device resources.

[0089] Exemplarily, when the image receiver determines that the image display mode is the first mode according to the current device environment, an image loading control, such as "Loading Picture", is generated according to the storage address corresponding to the target image, and text information, text recognition identifier and image loading control are displayed according to the preset layout. In response to the user's triggering operation on the image loading control, the image receiver obtains the updated device environment, and when the updated device environment meets the image display requirements for the target image, the target image is obtained from the server and displayed. When the updated device environment does not meet the image display requirements for the target image, image loading exception information is generated and displayed.

[0090] In this embodiment, by displaying the image loading control in the first mode, the user's demand for image display can be met, and the target image is acquired and displayed when the device environment is updated to meet the image display demand, thereby achieving the effectiveness of image transmission.

[0091] In an exemplary embodiment, the image transmission method further includes:

[0092] When the reading assistance function is enabled, the text information and the text recognition mark are converted into voice to obtain the text voice data corresponding to the target image, and the text voice data is played.

[0093] The reading assistance function refers to a function for reading image information provided by the image receiver.

[0094] Exemplarily, when the image receiver enables the reading assistance function, the image display mode corresponding to the target image is the first mode. The image receiver receives the text information and text recognition identifier sent by the server, displays the text information and text recognition identifier in the first mode, performs voice conversion on the text information and text recognition identifier, obtains text voice data corresponding to the target image, and reads the text information and text recognition identifier according to the text language data.

[0095] In this embodiment, by providing a reading assistance function, diversified display of image information can be achieved.

[0096] In an exemplary embodiment, Figure 4 As shown, a schematic diagram of image information display is provided. In the first mode, the image receiver displays text information and text recognition mark, such as Figure 4 The image information in the solid-line frame includes the text information corresponding to the text recognition mark and the loading image control. The image receiver obtains and updates the device environment in response to the user's triggering operation on the loading image control. When the device update environment meets the image display requirements, the target image is obtained from the server according to the image storage address of the target image and displayed. At this time, the image display model of the image receiver switches from the first mode to the second mode. In the second mode, the image receiver displays the text information, the text recognition mark and the target image, such as Figure 4 Image information within the dashed box.

[0097] In an exemplary embodiment, Figure 5 As shown, a schematic diagram of the process of image transmission is provided. The image sender selects an initial image to be sent in the instant messaging, encodes the initial image into a file of a fixed format, namely, an encoded image, and sends the encoded image to a backend server (hereinafter referred to as the server).

[0098] After receiving the encoded image, the server determines whether to re-encode the encoded image according to the encoding format of the image receiving method to obtain the target image, and then generates the unique loading credential information of the target image, that is, the storage address of the target image. The server is pre-set with text recognition models corresponding to different image types. The server identifies whether the target image is a pure text image (that is, the image type). If so, the text recognition identifier of the target image is determined to be a text recognition identifier, and the text recognition is performed on the target image to obtain the text information corresponding to the target image. The text recognition can be performed using an OCR model (Optical Character Recognition, optical character recognition) to perform text recognition on the target image, and the text recognition identifier is "OCR"; if not, the text recognition identifier of the target image is determined to be a model recognition identifier, and the target image is input into the target image-to-text model to obtain the text information corresponding to the target image. The target image-to-text model can be Mdjourney, Image2Caption, mPLUG image description model, BLIP-2 model, etc., and the text recognition identifier is "image-to-text".

[0099] The server sends the text information and text recognition mark of the target image to the image receiver. The image receiver receives the image information (text information and text recognition mark) and determines whether the image can be loaded in the receiver environment (current device environment). If not, the text information and text recognition mark are displayed in the first mode; if not, the text information, text recognition mark and the initial image after decoding the target image are displayed in the second mode. The image receiver displays the image information and determines whether the reading assistance function is enabled. If so, the image information is read aloud. After the reading is completed, the image receiver has completed receiving the image information; if not, the image receiver has completed receiving the image information.

[0100] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0101] Based on the same inventive concept, the embodiment of the present application also provides an image transmission device for implementing the above-mentioned image transmission method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image transmission device embodiments provided below can refer to the limitations on the image transmission method above, and will not be repeated here.

[0102] In an exemplary embodiment, Figure 6 As shown, an image transmission device 600 is provided, comprising: an acquisition module 602, a text recognition module 604 and an image sending module 606, wherein:

[0103] An acquisition module 602 is used to acquire a target image sent by an image sender;

[0104] A text recognition module 604 is used to determine a text recognition identifier of the target image based on the image type of the target image, and recognize text information of the target image based on the text recognition identifier;

[0105] The image sending module 606 is used to send text information and text recognition identifiers to the image receiver, so that the image receiver determines the image display mode based on the current device environment and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition identifiers, and the second mode is used to display text information, text recognition identifiers and target images.

[0106] In an exemplary embodiment, the text recognition module 604 is also used to determine the text recognition identifier of the target image as a text recognition identifier when the image type of the target image is a text image type, perform text recognition on the target image, and obtain text information corresponding to the target image; when the image type of the target image is a non-text image type, determine the text recognition identifier of the target image as a model recognition identifier, input the target image into a target image-to-text model, and obtain text information corresponding to the target image.

[0107] In an exemplary embodiment, the image transmission device 600 is also used to obtain a training image set corresponding to a non-text image type; determine a label dimension corresponding to each training image based on the image scene type corresponding to each training image in the training image set, annotate key information of each training image based on the label dimension, and obtain a training label set corresponding to the training image set; and perform supervised training on the initial image-to-text model based on the training image set and the training label set corresponding to the training image set to obtain a target image-to-text model.

[0108] In an exemplary embodiment, the acquisition module 602 is also used to obtain the encoded image sent by the image sender after encoding the initial image; when the encoding format corresponding to the image sender matches the decoding format corresponding to the image receiver, the encoded image is used as the target image; when the encoding format corresponding to the image sender does not match the decoding format of the image receiver, the encoded image is decoded to obtain the initial image; the current encoding format is determined based on the decoding format corresponding to the image receiver, and the initial image is re-encoded according to the current encoding format to obtain the target image.

[0109] In an exemplary embodiment, Figure 7 As shown, an image transmission device 700 is provided, including: an acquisition module 702 and a display module 704, wherein:

[0110] The acquisition module 702 is used to acquire text information and a text recognition identifier corresponding to the target image sent by the server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier;

[0111] The display module 704 is used to determine the image display mode according to the current device environment and display image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display text information and text recognition logo, and the second mode is used to display text information, text recognition logo and target image.

[0112] In an exemplary embodiment, the display module 704 is also used to display text information and text recognition identifiers based on a first mode when the current device environment does not meet the image display requirements for the target image; when the current device environment meets the image display requirements for the target image, obtain the target image from the server and display the text information, text recognition identifier and target image based on a second mode.

[0113] In an exemplary embodiment, the image transmission device 700 is also used to display an image loading control corresponding to a target image in a first mode; obtain an update device environment in response to a trigger operation on the image loading control; and obtain the target image from a server and display it when the update device environment meets the image display requirements for the target image.

[0114] In an exemplary embodiment, the image transmission device 700 is further used to perform voice conversion on the text information and the text recognition mark when the reading assistance function is enabled, obtain the text voice data corresponding to the target image, and play the text voice data.

[0115] Each module in the above-mentioned image transmission device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0116] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image transmission method is implemented.

[0117] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, an XXX method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0118] Those skilled in the art will understand that Figure 8-Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] In an exemplary embodiment, in one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0121] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0123] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0125] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image transmission method, characterized in that: The method comprises: Get the target image sent by the image sender; Determining a text recognition identifier of the target image based on an image type of the target image, and recognizing text information of the target image based on the text recognition identifier; The text information and the text recognition identifier are sent to an image receiver, so that the image receiver determines an image display mode based on the current device environment, and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition identifier, and the second mode is used to display the text information, the text recognition identifier and the target image.

2. The method according to claim 1, characterized in that The step of determining a text recognition identifier of the target image based on the image type of the target image, and recognizing text information of the target image based on the text recognition identifier includes: When the image type of the target image is a text image type, determining that the text recognition identifier of the target image is a character recognition identifier, performing character recognition on the target image, and obtaining text information corresponding to the target image; When the image type of the target image is a non-text image type, the text recognition identifier of the target image is determined to be a model recognition identifier, and the target image is input into a target image-to-text model to obtain text information corresponding to the target image.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining a training image set corresponding to the non-text image type; Based on the image scene types corresponding to the respective training images in the training image set, determining the label dimensions corresponding to the respective training images, and annotating the key information of the respective training images based on the label dimensions to obtain the training label set corresponding to the training image set; Based on the training image set and the training label set corresponding to the training image set, supervised training is performed on the initial image-to-text model to obtain a target image-to-text model.

4. The method according to claim 1, characterized in that The step of acquiring a target image sent by an image sender comprises: Obtaining a coded image sent by an image sender after encoding an initial image; When the encoding format corresponding to the image sender matches the decoding format corresponding to the image receiver, using the encoded image as the target image; When the encoding format corresponding to the image sender does not match the decoding format of the image receiver, decoding the encoded image to obtain the initial image; The current encoding format is determined based on the decoding format corresponding to the image receiver, and the initial image is re-encoded according to the current encoding format to obtain a target image.

5. An image transmission method, characterized in that: The method comprises: Acquire text information and a text recognition identifier corresponding to a target image sent by a server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier; An image display mode is determined according to the current device environment, and image information is displayed based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition mark, and the second mode is used to display the text information, the text recognition mark and the target image.

6. The method according to claim 5, characterized in that The determining of the image display mode according to the current device environment and displaying the image information based on the image display mode includes: When the current device environment does not meet the image display requirement for the target image, displaying the text information and the text recognition identifier based on the first mode; When the current device environment meets the image display requirement for the target image, the target image is obtained from the server, and the text information, the text recognition identifier and the target image are displayed based on the second mode.

7. The method according to claim 6, characterized in that The method further comprises: In the first mode, an image loading control corresponding to the target image is displayed; In response to a trigger operation on the image loading control, obtaining an update device environment; When the update device environment meets the image display requirement for the target image, the target image is obtained from the server and displayed.

8. The method according to claim 5, characterized in that The method further comprises: When the reading assistance function is enabled, the text information and the text recognition identifier are converted into voice to obtain text voice data corresponding to the target image, and the text voice data is played.

9. An image transmission device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image sent by an image sender; A text recognition module, used to determine a text recognition identifier of the target image based on the image type of the target image, and recognize text information of the target image based on the text recognition identifier; An image sending module is used to send the text information and the text recognition identifier to an image receiver, so that the image receiver determines an image display mode based on the current device environment and displays the image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition identifier, and the second mode is used to display the text information, the text recognition identifier and the target image.

10. An image transmission device, characterized in that: The device comprises: An acquisition module, used for acquiring text information and a text recognition identifier corresponding to a target image sent by a server; the text recognition identifier is determined by the server based on the image type of the target image sent by the image sender, and the text information is obtained by identifying the target image based on the text recognition identifier; A display module is used to determine an image display mode according to the current device environment and display image information based on the image display mode; the image display mode includes a first mode and a second mode, the first mode is used to display the text information and the text recognition mark, and the second mode is used to display the text information, the text recognition mark and the target image.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images