An image inference method, device, electronic equipment, medium and program product

Abstract

Embodiments of the present application disclose an image inference method and device, electronic equipment, medium and program product. When an image inference request of a to-be-inferred image is received, image identification information of the to-be-inferred image is obtained based on the image inference request, and a version number of a currently used image inference model is obtained. The image inference model is used to make recommendations according to the to-be-inferred image. According to the image identification information and the version number of the image inference model, a query identifier corresponding to the to-be-inferred image is generated. According to the query identifier, a target inference result matching the to-be-inferred image is obtained. The target inference result is used to determine recommended data matching the to-be-inferred image. This scheme can solve the problem of inaccurate image inference results and improve the accuracy of image inference results.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to an image reasoning method, apparatus, electronic device, medium, and program product. Background Technology

[0002] During video playback on video platforms, advertisements can be placed. Ad placement can be achieved by identifying video content to match the desired advertisements, thus enabling precise ad delivery. However, current image reasoning processes use only limited information for each received video, leading to inaccurate inference results and failing to meet expectations. Summary of the Invention

[0003] This invention provides an image reasoning method, apparatus, electronic device, medium, and program product, which can improve the accuracy of image reasoning results.

[0004] An image reasoning method, comprising:

[0005] Upon receiving an image reasoning request for an image to be reasoned, the image identification information of the image to be reasoned and the version number of the currently used image reasoning model are obtained based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned.

[0006] Based on the image identification information and the version number of the image reasoning model, a query identifier corresponding to the image to be reasoned is generated;

[0007] The target inference result matching the image to be inferred is obtained based on the query identifier, and the target inference result is used to determine the recommended data matching the image to be inferred.

[0008] Accordingly, embodiments of the present invention provide an image reasoning device, comprising:

[0009] The first acquisition unit is used to, upon receiving an image reasoning request for an image to be reasoned, acquire the image identification information of the image to be reasoned and the version number of the currently used image reasoning model based on the image to be reasoned; the image reasoning model is used to make recommendations based on the image to be reasoned.

[0010] The generation unit is used to generate a query identifier corresponding to the image to be inferred based on the image identification information and the version number of the image inference model.

[0011] The second acquisition unit is used to acquire a target inference result that matches the image to be inferred based on the query identifier, and the target inference result is used to determine recommended data that matches the image to be inferred.

[0012] Optionally, in some embodiments, the image reasoning device may further include a query unit. Specifically, the construction unit may be used to query a target reasoning result matching the image to be reasoned from an image reasoning database according to the query identifier. The image reasoning database stores reference reasoning results and reference query identifiers. The reference reasoning result is obtained by performing image reasoning on a reference reasoning image. The reference query identifier is generated based on the image identification information of the reference reasoning image and the version number of the reference image reasoning model. If no target reasoning result matching the image to be reasoned is found according to the query identifier, the version number of the image reasoning model is extracted from the query identifier. Image reasoning is then performed on the image to be reasoned using the image reasoning model corresponding to the version number to obtain the target reasoning result.

[0013] Optionally, in some embodiments, the query unit may be specifically used to output the target inference result if a target inference result matching the image to be inferred is found according to the query identifier.

[0014] Optionally, in some embodiments, the image inference device may further include an acquisition unit, which may be specifically used to acquire, upon receiving an image inference request for an image to be inferred, the identification information corresponding to the currently used image inference model; read the configuration information of the query identifier corresponding to the image inference model according to the identification information, wherein the configuration information is pre-set and synchronized to the image inference model via hot update; and acquire the image identification information of the image to be inferred and the version number of the currently used image inference model from the image inference request according to the configuration information, wherein the image inference model is used to make recommendations based on the image to be inferred.

[0015] Optionally, in some embodiments, the image reasoning device may further include a storage unit, which may be used to acquire a reference reasoning image; perform image reasoning on the reference reasoning image through a reference image reasoning model to obtain a reference reasoning result; associate and store the reference reasoning result with a reference query identifier corresponding to the reference reasoning result; and determine the target reasoning result from the reference reasoning result, wherein when the query identifier matches the reference query identifier, the target reasoning result matching the query identifier is determined from the reference reasoning result.

[0016] Optionally, in some embodiments, the image reasoning device may further include a reasoning unit, which may be used to extract feature information of the reference reasoning image, the feature information including at least one of text features, sound features and image features; identify the scene of the reference reasoning image based on the feature information; and perform image reasoning on the reference reasoning image based on the scene of the reference reasoning image to obtain a reference reasoning result.

[0017] Optionally, in some embodiments, the storage unit may be specifically used to upload the reference inference result to cloud storage; obtain the access address of the reference inference result in cloud storage; and associate the access address with the reference query identifier for storage.

[0018] Optionally, in some embodiments, the query unit may be specifically used to obtain the access address of the target inference result matching the image to be inferred in cloud storage according to the query identifier; and to obtain the target inference result matching the image to be inferred from cloud storage according to the access address.

[0019] Optionally, in some embodiments, the storage unit may be specifically used to obtain a cached value of the reference inference result; if the cached value is greater than or equal to a preset value, the reference inference result is uploaded to cloud storage; if the cached value is less than the preset value, the reference inference result is associated with and stored with the reference query identifier.

[0020] Furthermore, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the image reasoning method provided in embodiments of the present invention.

[0021] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of computer programs adapted for loading by a processor to execute steps in any of the image reasoning methods provided in embodiments of the present invention.

[0022] The image inference method, apparatus, electronic device, medium, and program product provided in this application embodiment allow the electronic device to, upon receiving an image inference request for an image to be inferred, obtain image identification information of the image to be inferred and the version number of the currently used image inference model based on the image inference request. The image inference model is used to make recommendations based on the image to be inferred. A query identifier corresponding to the image to be inferred is generated based on the image identification information and the version number of the image inference model. A target inference result matching the image to be inferred is obtained based on the query identifier. The target inference result is used to determine recommended data matching the image to be inferred. Therefore, by obtaining the version number of the currently used image inference model each time an image inference request is received for subsequent image inference, the online system can use the new version of the image inference model for image inference calculations after a new version of the image inference model is launched, thus avoiding the continued use of cached old version model inference results online after a model version update. Simultaneously, by using multiple information such as the image identification information of the image to be inferred and the version number of the currently used image inference model to generate the query identifier, the target inference result can be obtained more accurately based on the query identifier, improving the accuracy of the image inference result. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the image reasoning method provided in an embodiment of the present invention;

[0025] Figure 2 This is another schematic diagram of the image reasoning process provided in an embodiment of the present invention;

[0026] Figure 3 Application scenarios of the image reasoning method provided in this embodiment of the invention;

[0027] Figure 4 Application scenarios of the image reasoning method provided in this embodiment of the invention;

[0028] Figure 5 This is a diagram illustrating the overall architecture of the image reasoning system in this embodiment of the invention.

[0029] Figure 6 This is a schematic diagram of the image reasoning device provided in an embodiment of the present invention;

[0030] Figure 7This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This invention provides an image reasoning method, apparatus, and computer-readable storage medium. The image reasoning apparatus can be integrated into an electronic device, which may be a server or a terminal, etc.

[0033] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0034] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0035] This embodiment will be described from the perspective of an image reasoning device, which can be integrated into an electronic device, such as a server or a terminal. The terminal can include tablet computers, laptops, personal computers (PCs), wearable devices, virtual reality devices, or other electronic devices that can acquire data.

[0036] An image reasoning method, comprising:

[0037] Upon receiving an image reasoning request for an image to be reasoned, the system obtains the image identifier information of the image to be reasoned and the version number of the currently used image reasoning model based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned. Based on the image identifier information and the version number of the image reasoning model, a query identifier corresponding to the image to be reasoned is generated. Based on the query identifier, a target reasoning result matching the image to be reasoned is obtained. The target reasoning result is used to determine the recommended data matching the image to be reasoned.

[0038] like Figure 1 As shown, the specific process of this image reasoning method is as follows:

[0039] 101. Upon receiving an image reasoning request for an image to be reasoned, obtain the image identification information of the image to be reasoned and the version number of the currently used image reasoning model based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned.

[0040] In this embodiment, image reasoning via a model, relative to image training, refers to forward computation of the model. That is, given input data, the model calculates its output. Essentially, the image to be reasoned is input into the currently used image reasoning model, which then performs image reasoning to obtain the reasoning result. The image to be reasoned can be an image to be played on an electronic device. For example, the image to be reasoned can be one or more scattered pictures, or a video composed of multiple consecutive image frames. The image reasoning request can include image identification information of the image to be reasoned. This image identification information can include an image address, an image digital signature, and target configuration information. The image address can be the website address of the image or video in the image to be reasoned, or the location information of the image or video stored on the smart device. The image digital signature can be the MD5 value of the image content; when the image content changes, the image digital signature, i.e., the MD5 value, will also change. The target configuration information can be the identification information of the application software from which the image to be reasoned originates and / or the user identity information of the user logging into the application software or electronic device, or the identity information of the user who logged into the electronic device to obtain the image to be reasoned. This image reasoning model is used to perform image reasoning on the image to be reasoned, thereby obtaining a reasoning result. This result can be the content category corresponding to the image to be reasoned, or recommended content corresponding to the content category of the image to be reasoned. The image reasoning model can be an image recognition model. The version number of the image reasoning model can represent the currently used version; a change in the version number indicates a change in the version of the image reasoning model.

[0041] In this embodiment, the executing entity of the image reasoning method can be an electronic device, such as a mobile terminal like a mobile phone, iPad, tablet, or wearable device, or an electronic device like a smart screen or computer. The image to be reasoned can be pre-stored locally on the electronic device, or it can be downloaded locally from the network by an application software installed on the electronic device, or it can be obtained from a third-party server. In practical application scenarios, such as in an advertising system, image reasoning can be applied to short and long videos, using an image reasoning model to recommend highly relevant advertisements based on video content and plot information. The image to be reasoned can be a video played in the application software of the electronic device, the image address of the image to be reasoned can be the website address where the application software plays the video, the target configuration information can be the ID of the application software or the user identity information corresponding to the account logged into the application software, the currently used image reasoning model can be an algorithm model in the electronic device used to perform image recognition on videos in the application software and recommend advertisements, the version number of the currently used image reasoning model can be the latest version of the image reasoning model in the electronic device, and the reasoning result can be the image corresponding to an advertisement with high relevance to the content or plot information of the video to be reasoned, in order to determine which advertisement image to insert in which frame of the video.

[0042] For example, the electronic device can obtain the video website address of the video to be inferred, the identification information of the image source to be inferred, the user identity information of the user logged into the electronic device, and the version number of the image inference model currently used by the electronic device from local storage, online download via application software, or from a third-party server.

[0043] In one embodiment, upon receiving an image reasoning request for an image to be reasoned about, the image identification information of the image to be reasoned about and the version number of the currently used image reasoning model are obtained based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned about, and may include:

[0044] S1. Upon receiving an image reasoning request for an image to be reasoned, obtain the identification information corresponding to the currently used image reasoning model.

[0045] The image inference request can be triggered when an electronic device plays an image to be inferred. The image inference request can include the image identification information of the image to be inferred. The identification information corresponding to the image inference model is used to identify the image inference model. The user pre-binds this identification information to the image inference service corresponding to the image inference model. Based on this identification information, the configuration information of the query identifier corresponding to the currently used image inference model can be determined. This configuration information can be used to configure the fields that make up the query identifier.

[0046] For example, when an electronic device plays a video to be inferred, it triggers an image inference request corresponding to the video to obtain the identification information of the image inference model currently used by the electronic device.

[0047] S2. Read the configuration information of the query identifier corresponding to the image inference model based on the identification information. The configuration information is pre-set and synchronized to the image inference model through hot update.

[0048] The query identifier is used to retrieve the reference inference result cached by the electronic device. This identifier can be bound to the image inference model, allowing the electronic device to determine the version number of the image inference model based on the identifier, and then determine the query identifier based on the version number of the image inference model to retrieve the inference result corresponding to that image inference model. The configuration information indicates the information required to compose the query identifier. For example, if the configuration information includes the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model, then the query identifier can be composed of the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model. The reference inference result can be the content category corresponding to the reference inference image, or it can be the recommended content corresponding to the content category of the reference inference image. For example, in a practical application of advertising recommendation scenarios, the reference inference result can be the image corresponding to an advertisement with high relevance to the content or plot information of the reference inference video. The reference inference image can be one or more scattered images, or it can be a video composed of multiple consecutive image frames. The reference inference image can be an image or video inferred by the electronic device before receiving the image inference request for the image to be inferred, and the reference inference image has not been image inferred by the electronic device before obtaining the reference inference image. The image to be inferred acquired by the electronic device may be the same as or different from the reference inference image. The reference image inference model may be one or more, and the reference inference result may be one or more. If the image to be inferred is the same as the reference inference image, and the image model used for image inference on the reference inference image is the same as the current image inference model for the image to be inferred, then it means that the inference results corresponding to the image to be inferred and the reference inference image are the same.

[0049] For example, after obtaining the identification information of the currently used image inference model, the electronic device reads the configuration information of the query identifier corresponding to the currently used image inference model according to the identification information, so as to indicate the composition information of the query identifier.

[0050] S3. Based on the configuration information, obtain the image identification information of the image to be inferred from the image inference request and obtain the version number of the currently used image inference model.

[0051] If the configuration information can include image address, image digital signature, target configuration information, and version number of image inference model, then the electronic device can obtain the image identification information of the image to be inferred and the version number of the image inference model from the image inference request based on the configuration information, and thus combine the image identification information of the image to be inferred and the version number of the image inference model to form a query identifier.

[0052] For example, the image address of the image to be inferred in the configuration information can be the image address "img-url", the target configuration information can include "appid" and "userid", and the version number of the image inference model is "model-ver". Then, based on this configuration information, the image address of the image to be inferred can be obtained from the image inference request as "img-url: https: / / xxxxx.com / xxx". The target configuration information can be "appid: QQQ" and "userid: ddd", and the version number of the image inference model can be "model-ver: 1". Then, the query identifier can be "img-url: https: / / xxxxx.com / xxx", "appid: QQQ", "userid: ddd", and "model-ver: 1".

[0053] In this embodiment, when the electronic device plays an image to be inferred, it triggers an image inference request to obtain the identification information of the image inference model currently used by the electronic device. Based on this identification information, it reads the pre-defined configuration information of the query identifier corresponding to the currently used image inference model to indicate the composition information of the query identifier. If the configuration information includes the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model, then it means that the query identifier can be composed of the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model. Based on the configuration information, it obtains the image identification information of the image to be inferred and the version number of the image inference model from the image inference request. This allows the electronic device to read the pre-configured configuration information based on the identifier of the image inference model, enabling it to determine the content category of the image to be inferred and the version of the image inference model from the information obtained from the image to be inferred and the currently used image inference model.

[0054] In one embodiment, upon receiving an image reasoning request for an image to be reasoned about, the system obtains the image identification information of the image to be reasoned about and the version number of the currently used image reasoning model based on the image reasoning request. Before the image reasoning model makes recommendations based on the image to be reasoned about, it may further include:

[0055] S4. Obtain the reference inference image.

[0056] The reference inference image can be an image or video inferred by the electronic device before receiving the image inference request for the image to be inferred, and this reference inference image has not been inferred by the electronic device before being acquired. The image to be inferred acquired by the electronic device may be the same as or different from the reference inference image. When the image to be inferred and the reference inference image are the same, it means that the image identification information of the image to be inferred is the same as the reference image identification information of the reference inference image. That is, the image address of the image to be inferred is the same as the reference image address of the reference inference image, the image digital signature of the image to be inferred is the same as the reference image digital signature of the reference inference image, and the target configuration information of the image to be inferred is the same as the reference configuration information of the reference inference image.

[0057] It should be noted that, in this embodiment, in order to facilitate the electronic device to quickly output the target inference result corresponding to the image to be inferred, after the electronic device performs image inference on the reference inference image through the reference image inference model, it obtains the reference inference result and stores the reference inference result in association with the reference query identifier corresponding to the reference inference image and the reference image inference model. So that when the electronic device detects that the query identifier corresponding to the image to be inferred matches the stored reference query identifier, that is, when the image to be inferred is the same as the reference inference image previously inferred by the electronic device, and the image inference model currently used is the same as the reference image inference model used by the electronic device when inferring the reference inference image, it can directly obtain the target inference result corresponding to the query identifier from the cached reference inference result according to the query identifier. Thus, the target inference result can be obtained directly through the electronic device when the image to be inferred has already been inferred, without having to perform image inference again through the image inference model, thereby improving the efficiency of image inference of the electronic device.

[0058] The reference inference image can be pre-stored locally on the electronic device, downloaded online from the network by an application installed on the device, or obtained from a third-party server. The target inference result can be the inference result corresponding to the image to be inferred. This target inference result can be obtained from the reference inference result cached on the electronic device, or it can be inferred from an image inference model. In practical applications, such as in ad recommendation, the target inference result can be images of multiple frames of recommended ads corresponding to the content of the video currently playing on the electronic device and the scene information.

[0059] S5. Perform image reasoning on the reference reasoning image using the reference image reasoning model to obtain the reference reasoning result.

[0060] The reference image inference model can be a reference image inference model used by an electronic device to perform image inference on a reference inference image. The current image inference model of the electronic device may be the same as or different from the reference image inference model. The reference inference result can be the content category corresponding to the reference inference image, or it can be the recommended content corresponding to the content category of the reference inference image. The number of reference inference images, the number of reference image inference models, and the number of reference inference results can all be one or more. If the image to be inferred is the same as the reference inference image, and the image model for image inference on the reference inference image is the same as the current image inference model for the image to be inferred, then it means that the inference result corresponding to the image to be inferred and the reference inference image are the same, that is, the reference inference result is the same as the target inference result.

[0061] In this embodiment, inference, relative to training, refers to the forward computation of the model, that is, calculating the output result of the model for the given input data. This is equivalent to inputting the reference inference image into the reference image inference model, and then the reference image inference model obtains the reference inference result by performing image inference on the reference inference image.

[0062] In one embodiment, performing image reasoning on a reference reasoning image using a reference image reasoning model to obtain a reference reasoning result may include:

[0063] (1) Extract feature information from the reference inference image. The feature information includes at least one of text features, sound features and image features.

[0064] The image inference model can be a machine learning model in the field of images or videos, such as a PyTorch model. The reference inference image can be an image or video that has been previously inferred by the electronic device. The text feature can be the text feature in the text detection box corresponding to the subtitle text in the reference inference image. The sound feature can be the sound feature in the reference inference video. The image feature can be the image feature information such as the outline, shape, and pixels of the target object in the image or video. The target object can be a moving object or a target object that matches a pre-stored object.

[0065] For example, electronic devices extract text features and image features from reference inference images using reference image inference models.

[0066] (2) Identify the scene of the reference inference image based on the feature information.

[0067] Optionally, the electronic device can acquire a preset image labeling algorithm, match classification labels to feature information in the reference inference image using the preset image labeling algorithm, and then determine the scene of the reference inference image based on the classification labels in the reference inference image, thereby converting the visual information in the reference inference image into semantic information according to the classification labels.

[0068] The pre-defined image labeling algorithm converts visual information into semantic information by assigning appropriate labels to images, helping users better understand and analyze them. Image labeling includes single-class and multi-label image classification. Single-class image classification involves finding a matching label from a fixed set of labels for an input reference image and assigning it to that image. Multi-label image classification allows assigning multiple matching labels to a reference image to fully express its content.

[0069] Specifically, this image labeling algorithm can be implemented using a convolutional neural network (CNN). A CNN mainly consists of convolutional layers and pooling layers. The convolutional layers perform convolution with the entire image using a fixed-size filter; the pooling layers are a downsampling operation that takes the maximum and average values ​​of local blocks in the feature map obtained from the convolution to achieve downsampling. Non-linear activation functions are also added to the CNN to enhance its expressive power. At the end of the network, there can be several fully connected layers and a classifier. In image single-classification tasks, the input to the CNN can be a three-dimensional array of reference inference images, and the output can be the probability of each classification label predicted by the softmax classifier on the set of classification labels. The classification label with the highest score is the predicted category of the reference inference image. Then, the scene of the reference inference image is determined based on the classification labels in the reference inference image.

[0070] For example, if the text content obtained from recognizing text features is "It's raining, achoo," and classifying the text features in the reference inference image by matching classification labels to identify image features, and determining that the target object in the reference inference image is a person getting wet, then matching the image features in the reference inference image by matching classification labels to identify the scene in the reference inference image is a scene of a person getting wet and catching a cold. Similarly, if the extracted image features in the reference inference image determine that the target object is a moving car, then matching the image features in the reference inference image by matching classification labels to identify the scene in the reference inference image is a scene of a moving car.

[0071] (3) Perform image reasoning on the reference reasoning image based on the scene of the reference reasoning image to obtain the reference reasoning result.

[0072] It should be noted that the electronic device or the cloud server connected to the electronic device stores the matching relationships between different scenes corresponding to the reference inference image and the reference inference result. Optionally, the scene of the reference inference image is input into the reference image inference model, and the reference image inference model recognizes the reference inference image to obtain the scene matching reference inference result of the reference inference image from the local device or the server.

[0073] For example, if the scene identified in the reference inference image is a person getting wet and catching a cold, then based on this scene, the content category corresponding to the reference inference result of the reference inference image can be inferred to be the "illness" category. The reference inference result can also be recommended content corresponding to the illness category. This recommended content could be a link to a pre-stored cold medicine for sale or an official website for the illness category stored in the electronic device, as well as pre-stored comments corresponding to the illness category, such as "Is AAA sick? I feel sorry for him." Similarly, if the scene identified in the reference inference image is a car racing scene, then based on this scene, the content category corresponding to the reference inference result of the reference inference image can be inferred to be the "car" category. This reference inference result can also be recommended content corresponding to the car category. This recommended content could be a link to a pre-stored car for sale or an official website for the car category stored in the electronic device, as well as pre-stored comments corresponding to the car category, such as "New model launched, up to x% off."

[0074] S6. Associate and store the reference inference result with the reference query identifier corresponding to the reference inference result. The target inference result is determined from the reference inference result. When the query identifier matches the reference query identifier, the target inference result matching the query identifier is determined from the reference inference result.

[0075] In one embodiment, before associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result, the process may include: obtaining identification information corresponding to the reference image inference model; reading configuration information of the query identifier corresponding to the reference image inference model based on the identification information; obtaining the image address and reference configuration information of the reference inference image based on the configuration information, and obtaining the version number of the reference image inference model. Based on the reference image address, reference image digital signature, reference configuration information, and version number of the reference inference model, a reference query identifier corresponding to the reference inference image is generated.

[0076] It is understood that in this embodiment, the reference query identifier corresponding to the reference inference result is associated and saved with the reference inference result. Thus, when the electronic device detects that the query identifier corresponding to the image to be inferred and the current image inference model matches the reference query identifier, that is, when the image to be inferred and the reference inference image are the same, and the image inference model currently used is the same as the reference image inference model used by the electronic device when inferring the reference inference image before, the reference inference result can be obtained directly according to the query identifier without having to perform image inference on the image to be inferred again, which can improve the efficiency of image inference of the electronic device.

[0077] In one embodiment, associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result may include:

[0078] It should be noted that after obtaining the reference inference result by performing image inference on the reference inference image using the reference image inference model, the reference inference result can be stored in an image inference database. This image inference database can be a key-value-based data storage component. However, in this case, if the images or videos in the inference result occupy too much memory, the memory of the local image inference database of the electronic device will easily become full, resulting in poor storage performance of the inference result. Therefore, in this embodiment, the reference inference result can be uploaded to cloud storage, which can be the image inference database on a cloud server. Only the access address of the reference inference result in the cloud storage is associated with the reference query identifier corresponding to the reference inference result and stored in the local image inference database, thereby improving the storage performance of the image inference result of the electronic device. Specifically, it can be as follows:

[0079] (1) Upload the reference reasoning results to cloud storage.

[0080] It should be noted that cloud storage is a new concept that has been extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file systems to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) to work together through application software or application interfaces to provide data storage and business access functions to the outside world.

[0081] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0082] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0083] In this embodiment, cloud storage can be object cloud storage. The reference inference result can be the content category corresponding to the reference inference image, or it can be recommended content corresponding to the content category of the reference inference image.

[0084] For example, after performing image reasoning on a reference inference image using a reference image reasoning model, an electronic device obtains a reference reasoning result and uploads the reference reasoning result to an object cloud storage.

[0085] (2) Obtain the access address of the reference inference result in cloud storage.

[0086] For example, after an electronic device uploads the reference inference results to cloud storage, it accesses the reference inference results in the cloud storage to obtain the access address of the reference inference results in the cloud storage.

[0087] (3) Associate the access address with the reference query identifier and store it.

[0088] For example, the access address of the reference inference result in cloud storage can be associated with the reference query identifier corresponding to the reference inference result and stored in the local image inference database. This makes it convenient for electronic devices to retrieve the target inference result corresponding to the query identifier from the local image inference database when they detect a query identifier that matches the reference query identifier corresponding to the reference inference result.

[0089] Correspondingly, obtaining the target inference result matching the image to be inferred based on the query identifier may include: obtaining the access address of the target inference result matching the image to be inferred in cloud storage based on the query identifier; and obtaining the target inference result matching the image to be inferred from cloud storage based on the access address.

[0090] For example, a target inference result matching the image to be inferred is queried from the local image inference database based on the query identifier. If a target inference result matching the image to be inferred is found based on the query identifier, the access address of the target inference result in the cloud storage is obtained from the local image inference database based on the query identifier, and the target inference result is obtained from the cloud storage based on the access address.

[0091] In one embodiment, uploading the reference inference result to cloud storage may include:

[0092] (4) Obtain the cached value of the reference inference result.

[0093] The cache value can refer to the memory occupied by the reference inference result. The larger the cache value, the larger the memory occupied by the reference inference result. For example, if the reference inference result is an image, then the larger the cache value of the reference inference result, the more images or the larger the image pixels corresponding to the reference inference result.

[0094] For example, after an electronic device performs image reasoning on a reference inference image using a reference image reasoning model and obtains a reference inference result, it retrieves the cached value of that reference inference result.

[0095] (5) If the cached value is greater than or equal to the preset value, the reference inference result will be uploaded to the cloud storage.

[0096] The preset values ​​can be user-defined, set at the factory, or set in the application software used for image reasoning within the electronic device.

[0097] For example, it determines whether the cached value of the reference inference result is greater than or equal to a preset value. If the cached value is greater than or equal to the preset value, the reference inference result is uploaded to cloud storage, and the access address of the reference inference result in cloud storage is obtained. Then, the access address is associated with the reference query identifier and stored in the local image inference database. Correspondingly, obtaining the target inference result matching the image to be inferred based on the query identifier may include: obtaining the access address of the target inference result matching the image to be inferred in cloud storage based on the query identifier; and obtaining the target inference result matching the image to be inferred from cloud storage based on the access address.

[0098] In one embodiment, associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result may include:

[0099] (6) If the cached value is less than the preset value, the reference inference result will be associated with the reference query identifier and stored.

[0100] For example, if the cached value of the reference inference result is less than a preset value, the reference inference result is associated with the reference query identifier and stored in the local image inference database. Correspondingly, obtaining the target inference result that matches the image to be inferred based on the query identifier may include: querying the local image inference database based on the query identifier and obtaining the reference inference result associated with the query identifier as the target inference result.

[0101] 102. Generate a query identifier corresponding to the image to be inferred based on the image identification information and the version number of the image inference model.

[0102] The image identification information may include an image address, an image digital signature, and target configuration information. The image address may be the website address of the image or video in the image to be inferred, or the location information of the image or video stored on the smart device. The image digital signature may be the MD5 value of the image content; this MD5 value changes when the image content changes. The target configuration information may be the identification information of the application software from which the image to be inferred originates, and / or the user identity information of the user logging into the application software or electronic device, or the identity information of the user who logged into the electronic device to obtain the image to be inferred. The image inference model is used to perform image inference on the image to be inferred, thereby obtaining the inference result. This inference result may be the content category corresponding to the image to be inferred, or recommended content corresponding to the content category of the image to be inferred. The image inference model may be an image recognition model. The version number of the image inference model may represent the currently used version of the image inference model; an updated version number indicates an updated version of the image inference model.

[0103] For example, the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model are concatenated to form the query identifier corresponding to the image to be inferred. If the obtained image address of the image to be inferred is "img-url: https: / / xxxxx.com / xxx", the target configuration information is "appid: QQQ" and "userid: ddd", and the version number of the image inference model is "model-ver: 1", then the query identifier corresponding to the image to be inferred can be "img-url: https: / / xxxxx.com / xxx", "appid: QQQ", "userid: ddd", and "model-ver: 1".

[0104] 103. Obtain the target inference result that matches the image to be inferred based on the query identifier. The target inference result is used to determine the recommended data that matches the image to be inferred.

[0105] The query identifier is generated based on the image identification information of the image to be inferred and the version number of the current image inference model in the electronic device. This query identifier is used to determine the reference inference result matched by the reference query identifier as the target inference result when the query identifier matches the reference query identifier. The target inference result can be the inference result corresponding to the image to be inferred. This target inference result can be obtained from reference inference results cached by the electronic device, or it can be inferred based on the image inference model. Correspondingly, the recommended target inference result can be the display of the content category or scene information of the image to be inferred on the electronic device's display interface, or it can be the display of recommended content corresponding to the content category of the image to be inferred on the electronic device's display interface, such as advertisements and bullet comments. If the target inference result is the content category or scene information of the image to be inferred, then the recommended data matched by the image to be inferred determined by the target inference result can be advertisement data or bullet comment data, etc.

[0106] For example, the reference inference result that matches the query identifier can be obtained from the reference inference result as the target inference result, or the target inference result can be obtained by performing image inference on the image to be inferred through the image inference model based on the query identifier, and the recommended data corresponding to the target inference result of the image to be inferred, such as advertisements and bullet comments, can be displayed on the display interface of the electronic device.

[0107] In one embodiment, obtaining the target inference result matching the image to be inferred based on the query identifier may include:

[0108] S1. Query the target reasoning result that matches the image to be reasoned from the image reasoning database according to the query identifier. The image reasoning database stores reference reasoning results and reference query identifiers. The reference reasoning results are obtained by performing image reasoning on the reference reasoning image. The reference query identifier is generated based on the image identification information of the reference reasoning image and the version number of the reference image reasoning model.

[0109] The query identifier refers to the query identifier corresponding to the image to be inferred. The image inference database can be set in an electronic device or in a cloud server connected to the electronic device. The image inference database can include associated stored reference inference results and reference query identifiers.

[0110] For example, an electronic device can compare a query identifier with a reference query identifier corresponding to a reference inference result stored in an image inference database to determine whether the reference query identifier corresponding to the reference inference result matches the query identifier corresponding to the image to be inferred. If they match, it means that a target inference result matching the image to be inferred has been found based on the query identifier.

[0111] S2. If a target reasoning result matching the image to be reasoned is found based on the query identifier, then the target reasoning result is output.

[0112] Optionally, if the query identifier corresponding to the image to be inferred is the same as the reference query identifier corresponding to the reference inference result, then it is determined that the query identifier matches the reference query identifier, that is, a target inference result matching the image to be inferred is found based on the query identifier. If a target inference result matching the image to be inferred is found based on the query identifier, it means that the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred are all the same as those of the reference image address, image digital signature, target configuration information, and version number of the reference image inference model of the reference inference image.

[0113] For example, if the reference inference results cached by the electronic device include X, Y and Z, the reference query identifier corresponding to reference inference result X is L, the reference query identifier corresponding to reference inference result Y is M, and the reference query identifier corresponding to reference inference result Z is N, then if the query identifier corresponding to the image to be inferred is M, then the target inference result matching the image to be inferred is found according to the query identifier M, and this target inference result is the reference inference result Y.

[0114] Currently, online inference services for image models are characterized by numerous repetitive requests, high request processing latency, and high resource consumption. The image inference method provided in this embodiment can quickly query the corresponding target inference result when an image inference request is received, by using the corresponding query identifier. This allows the corresponding target inference result to be directly retrieved if it has been cached previously, without having to repeat the inference calculation, thus improving the efficiency of image inference.

[0115] In one embodiment, after retrieving a target inference result matching the image to be inferred from the image inference database based on a query identifier, the image inference method may further include:

[0116] S3. If no target reasoning result matching the image to be reasoned is found based on the query identifier, then extract the version number of the image reasoning model from the query identifier.

[0117] It should be noted that in this embodiment, if no matching target inference result is found in the image inference database based on the query identifier, meaning that the query identifier of the image to be inferred is not the same as the reference query identifier cached in the electronic device, and since the query identifier is generated based on the image address, image digital signature, target configuration information, and version number of the image inference model of the image to be inferred, it means that at least one of the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred is different from at least one of the reference image address, reference image digital signature, reference configuration information, and version number of the reference image inference model of the reference inference image. Therefore, the target inference result cannot be directly obtained from the reference inference results cached by the electronic device, and it is necessary to perform image inference on the image to be inferred using the current image inference model to obtain the target inference result.

[0118] Optionally, if the query identifier corresponding to the image to be inferred is different from the reference query identifier corresponding to the reference inference result, it is determined that the query identifier and the reference query identifier do not match, that is, no target inference result matching the image to be inferred is found based on the query identifier.

[0119] For example, if the reference inference results cached by the electronic device include X, Y, and Z, and the reference query identifier corresponding to reference inference result X is L, the reference query identifier corresponding to reference inference result Y is M, and the reference query identifier corresponding to reference inference result Z is N, then if the query identifier corresponding to the image to be inferred is S, then no target inference result matching the image to be inferred is found according to the query identifier S. In this case, the version number of the image inference model is extracted from the query identifier so as to determine the image inference model currently used by the electronic device based on the version number, and the image inference model is used to perform image inference on the image to be inferred to obtain the target inference result.

[0120] S4. Perform image reasoning on the image to be reasoned using the image reasoning model corresponding to the version number to obtain the target reasoning result.

[0121] Optionally, feature information of the image to be inferred can be extracted by querying the image inference model corresponding to the version number in the identifier. Based on the feature information, the content of the image to be inferred can be identified. Based on the content of the image to be inferred, image inference can be performed on the image to be inferred to obtain the target inference result.

[0122] Optionally, after performing image reasoning on the image to be reasoned using the version number-corresponding image reasoning model to obtain the target reasoning result, the method may further include: associating and storing the target reasoning result with a query identifier. This allows the electronic device to directly retrieve the target reasoning result when it detects a new query identifier that matches the query identifier.

[0123] Optionally, in one embodiment, obtaining the target reasoning result matching the image to be reasoned based on the query identifier may further include: extracting the version number of the image reasoning model from the query identifier; and performing image reasoning on the image to be reasoned using the image reasoning model corresponding to the version number to obtain the target reasoning result.

[0124] For example, an electronic device obtains the image identification information of the image to be inferred and the version number of the currently used image inference model. The image inference model is used to make recommendations based on the image to be inferred. The currently used image inference model can be determined directly based on the version number of the currently used image inference model. Image inference is performed on the image to be inferred through the image inference model to obtain the target inference result, and then the target inference result is used to recommend advertisements to the image to be inferred.

[0125] Specific implementation examples are the same as those described above, and will not be repeated here.

[0126] In this embodiment, if no target inference result matching the image to be inferred is found based on the query identifier, it indicates that at least one of the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred is different from at least one of the reference image address, reference image digital signature, reference configuration information, and version number of the reference image inference model of the reference inference image. Therefore, the target inference result cannot be directly obtained from the reference inference result cached by the electronic device. Instead, image inference of the image to be inferred needs to be performed using the current image inference model to obtain the target inference result. In other words, if the version number of the currently used image inference model is different from the version number of the reference image inference model, the reference inference result corresponding to the reference image inference model cannot be used. Only the currently used image inference model can be used to perform image inference of the image to be inferred to obtain the target inference result. Generally speaking, the currently used image inference model is an updated version compared to the reference image inference model. This ensures that after the new version of the image inference model is launched, all images to be inferred in the online system can use the new version of the image inference model for image inference calculation. This avoids the situation where some images or videos use the old version of the model for inference results, while some images or videos use the new version of the model for inference results, thus improving the image inference effect.

[0127] As described above, in this embodiment of the invention, the electronic device can obtain the image identification information of the image to be inferred and the version number of the currently used image inference model; generate a query identifier corresponding to the image to be inferred based on the image identification information and the version number of the image inference model; and obtain the target inference result matching the image to be inferred based on the query identifier. Therefore, by obtaining the version number of the currently used image inference model each time an image inference request is received for subsequent image inference, the online system can use the new version of the image inference model for image inference calculations after a new version is released, thus avoiding the continued use of cached old version model inference results online after a model version update. Simultaneously, by using multiple information such as the image identification information of the image to be inferred and the version number of the image inference model to generate the query identifier, the target inference result can be obtained more accurately based on the query identifier, improving the accuracy of the image inference result.

[0128] Based on the method described in the above embodiments, the following examples will provide further detailed explanations.

[0129] In this embodiment, the image reasoning device is specifically integrated into an electronic device. In this embodiment, the electronic device is a mobile phone, and the image to be reasoned is a picture or video played by an application software on the mobile phone.

[0130] like Figure 2 As shown, an image reasoning method has the following specific process:

[0131] 201. Electronic devices acquire reference inference images.

[0132] The reference inference image can be one or more discrete images, or a video composed of multiple consecutive image frames. This reference inference image can be an image or video inferred by the electronic device before receiving the image inference request for the image to be inferred, and it has not been inferred by the electronic device before being acquired. The image to be inferred acquired by the electronic device may or may not be the same as the reference inference image. In this embodiment, image inference refers to model forward computation, that is, calculating the model's output result for the given input data. This is equivalent to inputting the image to be inferred into the currently used image inference model, which then performs image inference on the image to be inferred to obtain the inference result.

[0133] The reference inference image can be pre-stored locally on the electronic device, downloaded online from the network by an application installed on the device, or obtained from a third-party server. The electronic device acquires the reference inference image upon receiving a request to play it. The image to be inferred is a newly acquired image after the electronic device obtains the reference inference result by performing image inference on the reference inference image. The image to be inferred can be one or more scattered images, or a video composed of multiple consecutive image frames. This image or video can be a television program or an advertising video. The image to be inferred can be the same as or different from the reference inference image. If the image to be inferred is the same as the reference inference image, it means that the content and playback source of the new image to be inferred are the same as those of previously inferred reference inference images. If the image to be inferred is different from the image to be inferred, it means that the content and playback source of the new image to be inferred are completely new and different from those of previously inferred reference inference images. The image playback source can be an image address.

[0134] In practical applications, such as advertising systems, image reasoning can be applied to short and long videos. By using an image reasoning model, it can recommend highly relevant advertisements based on video content and plot information. The reference reasoning image can be a video previously played in the application software of an electronic device. The image address of the reference reasoning image can be the website address where the application software plays the video. The target configuration information can be the ID of the application software that plays the reference reasoning image or the user identity information corresponding to the account that logs into the application software. The reference image reasoning model can be an algorithm model in the electronic device used to perform image recognition on the video in the application software and recommend advertisements.

[0135] 202. Electronic devices perform image reasoning on reference reasoning images through a reference image reasoning model to obtain reference reasoning results.

[0136] In this embodiment, image reasoning, relative to training, refers to the forward computation of the model. That is, given input data, the model's output is calculated. This is equivalent to inputting a reference image into a reference image reasoning model, which then performs image reasoning on the reference image to obtain a reference reasoning result. This reference image reasoning model can be a reference image reasoning model used by the electronic device to perform image reasoning on the reference image. The reference reasoning result can be the content category corresponding to the reference image, or recommended content corresponding to the content category of the reference image. The current image reasoning model of the electronic device may be the same as or different from the reference image reasoning model. There can be one or more reference image reasoning models, and there can be one or more reference reasoning results. If the image to be reasoned is the same as the reference image, and the image reasoning model currently used by the electronic device is the same as the reference image, then it means that the reasoning results corresponding to the image to be reasoned and the reference image are the same.

[0137] In one embodiment, performing image reasoning on a reference reasoning image using a reference image reasoning model to obtain a reference reasoning result may include:

[0138] (1) Extract feature information from the reference inference image. The feature information includes at least one of text features, sound features and image features.

[0139] The image inference model can be a machine learning model in the field of images or videos, such as a PyTorch model. The reference inference image can be an image or video that has been previously inferred by the electronic device. The text feature can be the text feature in the text detection box corresponding to the subtitle text in the reference inference image. The sound feature can be the sound feature in the reference inference video. The image feature can be the image feature information such as the outline, shape, and pixels of the target object in the image or video. The target object can be a moving object or a target object that matches a pre-stored object.

[0140] For example, electronic devices extract text features and image features from reference inference images using reference image inference models.

[0141] (2) Identify the scene of the reference inference image based on the feature information.

[0142] Reference Figure 3 and Figure 4 , Figure 3 and Figure 4The image reasoning method provided in this embodiment of the invention is used in the following application scenario: when playing a video or image in a specific application software on a mobile device, the electronic device extracts feature information of the reference reasoning image through a reference image reasoning model, identifies the scene of the reference reasoning image based on the feature information, and performs image reasoning on the reference reasoning image based on the scene to obtain the recommended content corresponding to the reference reasoning image.

[0143] Optionally, the electronic device can acquire a preset image labeling algorithm, match classification labels to feature information in the reference inference image using the preset image labeling algorithm, and then determine the scene of the reference inference image based on the classification labels in the reference inference image, thereby converting the visual information in the reference inference image into semantic information according to the classification labels.

[0144] The pre-defined image labeling algorithm converts visual information into semantic information by assigning appropriate labels to images, helping users better understand and analyze them. Image labeling includes single-class and multi-label image classification. Single-class image classification involves finding a matching label from a fixed set of labels for an input reference image and assigning it to that image. Multi-label image classification allows assigning multiple matching labels to a reference image to fully express its specific content.

[0145] Specifically, this image labeling algorithm can be implemented using a convolutional neural network (CNN). A CNN mainly consists of convolutional layers and pooling layers. The convolutional layers perform convolution with the entire image using a fixed-size filter; the pooling layers are a downsampling operation that takes the maximum and average values ​​of local blocks in the feature map obtained from the convolution to achieve downsampling. Non-linear activation functions are also added to the CNN to enhance its expressive power. At the end of the network, there can be several fully connected layers and a classifier. In image single-classification tasks, the input to the CNN can be a three-dimensional array of reference inference images, and the output can be the probability of each classification label predicted by the softmax classifier on the set of classification labels. The classification label with the highest score is the predicted category of the reference inference image. Then, the scene of the reference inference image is determined based on the classification labels in the reference inference image.

[0146] For example, refer to Figure 3 As shown, if the extracted image features indicate that the target object in the reference inference image is a moving car, then the scene identified in the reference inference image is a car motion scene. Figure 4As shown, if the text content obtained by recognizing text features is "cough cough cough", and the target object in the reference inference image obtained by recognizing image features is a person who is wet, then the scene of recognizing the reference inference image is a scene of a person who is wet and has a cold.

[0147] (3) Perform image reasoning on the reference reasoning image based on the scene of the reference reasoning image to obtain the reference reasoning result.

[0148] Optionally, the scene of the reference inference image is input into the reference image inference model so that the scene of the reference inference image is matched with the reference inference result through the reference image inference model.

[0149] For example, refer to Figure 4 If the scene identified in the reference inference image is a person getting wet and catching a cold, then based on this scene, the content category corresponding to the reference inference result of the reference inference image can be inferred to be the "illness" category. The reference inference result can also be recommended content corresponding to the illness category. This recommended content could be a link to a pre-stored cold medicine for that illness category on an electronic device, or a pre-stored comment for that illness category, such as "Is AAA sick? I feel sorry for him." Figure 3 If the scene identified in the reference inference image is a car motion scene, then based on the car motion scene, the content category corresponding to the reference inference result of the reference inference image can be inferred to be the car category. This reference inference result can also be recommended content corresponding to the car category. This recommended content can be a car sales link or official website link corresponding to the car category pre-stored in the electronic device, as well as pre-stored pop-up comments corresponding to the car category, such as "New model launched, up to x% off". If a user clicks on the sales link based on the displayed recommended content, the user can be redirected to the sales interface of the recommended content.

[0150] 203. Electronic devices associate and store the reference inference results with the reference query identifiers corresponding to the reference inference results.

[0151] The reference query identifier can be a cache key. After the reference query identifier is associated with and saved with the reference inference result, it can be used to query the reference inference result.

[0152] It is understood that in this embodiment, the reference query identifier corresponding to the reference inference result is associated and saved with the reference inference result. Thus, when the electronic device detects that the query identifier corresponding to the image to be inferred and the current image inference model matches the reference query identifier, that is, when the image to be inferred and the reference inference image are the same, the reference inference result can be obtained directly according to the query identifier without having to perform image inference on the image to be inferred again, which can improve the efficiency of image inference of the electronic device.

[0153] In one embodiment, associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result may include:

[0154] It should be noted that in existing technologies, after obtaining the reference inference result by performing image inference on the reference inference image using a reference image inference model, the reference inference result is generally stored in a local image inference database. This local image inference database can be a key-value-based data storage component. When storing the reference inference result in a local image inference database, if the images or videos in the inference result occupy too much memory, the local image inference database of the electronic device can easily become full, resulting in poor storage performance. Therefore, in this embodiment, the reference inference result can be uploaded to cloud storage, and then only the access address of the reference inference result in the cloud storage is associated with the reference query identifier corresponding to the reference inference result and stored in the local image inference database, thereby improving the storage performance of the image inference result of the electronic device. Specifically, it can be as follows:

[0155] (1) Upload the reference reasoning results to cloud storage.

[0156] Here, cloud storage can be object cloud storage. The reference inference result can be the content category corresponding to the reference inference image, or it can be recommended content corresponding to the content category of the reference inference image.

[0157] For example, after performing image reasoning on a reference inference image using a reference image reasoning model, an electronic device obtains a reference reasoning result and uploads the reference reasoning result to an object cloud storage.

[0158] (2) Obtain the access address of the reference inference result in cloud storage.

[0159] For example, after an electronic device uploads the reference inference results to cloud storage, it accesses the reference inference results in the cloud storage to obtain the access address of the reference inference results in the cloud storage.

[0160] (3) Associate the access address with the reference query identifier and store it.

[0161] The access address and reference query identifier are associated and stored in a local image reasoning database, which can be stored in a Redis remote dictionary server.

[0162] For example, the access address of the reference inference result in cloud storage can be associated with the reference query identifier corresponding to the reference inference result and stored in the local image inference database. This makes it convenient for electronic devices to retrieve the target inference result corresponding to the query identifier from the local image inference database when they detect a query identifier that matches the reference query identifier corresponding to the reference inference result.

[0163] The target inference result can be the inference result corresponding to the image to be inferred. This target inference result can be obtained from reference inference results cached by the electronic device, or it can be inferred based on an image inference model. In practical applications, such as advertising systems, the target inference result can be images of multi-frame recommended advertisements corresponding to the content and scene information of the video currently being played by the electronic device. This target inference result is used to determine which advertisement image the electronic device should insert in which frame of the video.

[0164] Correspondingly, obtaining the target inference result matching the image to be inferred based on the query identifier may include: obtaining the access address of the target inference result matching the image to be inferred in cloud storage based on the query identifier; and obtaining the target inference result matching the image to be inferred from cloud storage based on the access address.

[0165] For example, a target inference result matching the image to be inferred is queried from the image inference database based on the query identifier. If a target inference result matching the image to be inferred is found based on the query identifier, the access address of the target inference result in the cloud storage is obtained from the local image inference database based on the query identifier, and the target inference result is obtained from the cloud storage based on the access address.

[0166] Reference Figure 5 , Figure 5 This is an overall architecture diagram of the image inference system in this embodiment. When the image inference model receives an image inference request for a reference image, it performs image inference on the reference image and outputs a reference inference result. This reference inference result can be an image or video file. After receiving the reference inference result, the general caching service uploads it to the object cloud storage. After uploading, the object cloud storage accesses the access address of the reference inference result and associates the access address with the reference inference result, storing it in its local image inference database.

[0167] Optionally, in one embodiment, the reference inference result is associated and stored with the reference query identifier corresponding to the reference inference result, including:

[0168] (4) Obtain the cached value of the reference inference result.

[0169] The cache value can refer to the memory occupied by the reference inference result. The larger the cache value, the larger the memory occupied by the reference inference result. For example, if the reference inference result is an image, then the larger the cache value of the reference inference result, the more images or the larger the image pixels corresponding to the reference inference result.

[0170] For example, after an electronic device performs image reasoning on a reference inference image using a reference image reasoning model and obtains a reference inference result, it retrieves the cached value of that reference inference result.

[0171] (5) If the cached value is greater than or equal to the preset value, the reference inference result will be uploaded to the cloud storage;

[0172] The preset values ​​can be user-defined, set at the factory, or set in the application software used for image reasoning within the electronic device.

[0173] For example, it determines whether the cached value of the reference inference result is greater than or equal to a preset value. If the cached value is greater than or equal to the preset value, the reference inference result is uploaded to cloud storage, and the access address of the reference inference result in cloud storage is obtained. Then, the access address is associated with the reference query identifier and stored in the local image inference database. Correspondingly, obtaining the target inference result matching the image to be inferred based on the query identifier may include: obtaining the access address of the target inference result matching the image to be inferred in cloud storage based on the query identifier; and obtaining the target inference result matching the image to be inferred from cloud storage based on the access address.

[0174] (6) If the cached value is less than the preset value, the reference inference result will be associated with the reference query identifier and stored.

[0175] For example, if the cached value of the reference inference result is less than a preset value, the reference inference result is associated with the reference query identifier and stored in the local image inference database. Correspondingly, obtaining the target inference result that matches the image to be inferred based on the query identifier may include: querying the local image inference database based on the query identifier and obtaining the reference inference result associated with the query identifier as the target inference result.

[0176] Optionally, after storing the reference inference result associated with the reference query identifier, the storage identifier information is used to distinguish the content type stored in the local image inference database. This content type can be the reference query identifier and the reference inference result, or the access address of the reference query identifier and the reference inference result in cloud storage.

[0177] The storage identifier information can be a `type` field, such as "Redis-value". "0" can represent the reference query identifier and reference inference result stored in `value`, and "1" can represent the access address of the reference query identifier and reference inference result stored in `value` in cloud storage. For example, "Redis-value": "type": 1, "value": "https: / / xxx.cos.com / xxx.jpg", indicates that the access address of the reference query identifier and reference inference result stored in `value` in cloud storage is "https: / / xxx.cos.com / xxx.jpg".

[0178] In this embodiment, the electronic device directly associates reference inference results with small cache values ​​with reference query identifiers and stores them in the local image inference database, thereby achieving fast storage. For reference inference results with excessively large cache values, they can be uploaded to the cloud, and the access address of the reference inference result in the cloud can be associated with the reference query identifier and stored in the local image inference database. This effectively alleviates the pressure on the local image inference database, thereby reducing the storage time of reference inference results and improving the image inference efficiency of the electronic device.

[0179] 204. When an electronic device receives an image reasoning request for an image to be reasoned, it obtains the image identification information of the image to be reasoned and the version number of the currently used image reasoning model based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned.

[0180] The image to be inferred is the image currently being inferred by the electronic device. This image can be one or more scattered pictures, or a video composed of multiple consecutive image frames. The picture or video can be a television program or an advertising video. The image inference request can include image identification information of the image to be inferred, which may include an image address, an image digital signature, and target configuration information. The image address can be the website address of the picture or video in the image to be inferred, or the location information of the image or video stored on the smart device. The image digital signature can be the MD5 value of the image content; when the image content changes, the image digital signature, i.e., the MD5 value, will also change. The target configuration information can be the identification information of the application software from which the image to be inferred originates and / or the user identity information of the user logging into the application software or electronic device, or the identity information of the user who logged into the electronic device to obtain the image to be inferred. The image inference model is used to perform image inference on the image to be inferred, thereby obtaining the inference result. The inference result can be the content category corresponding to the content category of the image to be inferred, or recommended content corresponding to the content category of the image to be inferred. The image inference model can be an image recognition model. The version number of the image inference model can represent the version of the image inference model currently in use. If the version number changes, it means that the version of the image inference model has changed.

[0181] In this embodiment, the image to be inferred can be pre-stored locally on the electronic device, or it can be downloaded locally from the network by the application software installed on the electronic device, or it can be obtained by the electronic device from a third-party server. In practical application scenarios, such as advertising systems, image inference can be applied to short and long videos, using an image inference model to recommend highly relevant advertisements based on video content and plot information. The image to be inferred can be a video played in the application software of the electronic device, the image address of the image to be inferred can be the website address where the application software plays the video, the target configuration information can be the ID of the application software or the user identity information corresponding to the account logged into the application software, the currently used image inference model can be the algorithm model in the electronic device used to perform image recognition on the video in the application software and recommend advertisements, and the version number of the currently used image inference model can be the version number of the latest version of the image inference model in the electronic device.

[0182] For example, the electronic device obtains from the local video website address of the video to be inferred, the identification information of the image source to be inferred, the user identity information of the user who logged into the electronic device, and the version number of the image inference model currently used by the electronic device.

[0183] In one embodiment, obtaining the image identification information of the image to be inferred, and obtaining the version number of the currently used image inference model, may include:

[0184] S1. Upon receiving an image reasoning request, obtain the identification information corresponding to the currently used image reasoning model.

[0185] The image inference request can be triggered when an electronic device plays an image to be inferred. The image inference request can include the image identification information of the image to be inferred. The identification information corresponding to the image inference model is used to identify the image inference model. The user pre-binds the identification information to the image inference model. Based on the identification information, the configuration information of the query identifier corresponding to the currently used image inference model can be determined. This configuration information can be used to configure the fields that make up the query identifier.

[0186] For example, when an electronic device plays an image to be inferred, it triggers an image inference request to obtain the identification information of the image inference model currently used by the electronic device.

[0187] S2. Read the configuration information of the query identifier corresponding to the image inference model based on the identification information. The configuration information is pre-set and synchronized to the image inference model through hot update.

[0188] The query identifier can be a cache key, used to retrieve the inference result corresponding to the image inference model cached by the electronic device. This identifier can be bound to the image inference model, allowing the electronic device to determine the version number of the image inference model based on this identifier, and then determine the query identifier based on the version number to retrieve the inference result corresponding to that image inference model. The configuration information is used to configure the information required to form the query identifier.

[0189] For example, the query identifier can be configured as "model-1": "url": "img-url", "model-ver": "1", "other-input-param": "appid", "userid". The "url" field can be either "img-url" or "model-ver", indicating whether the image address to be inferred is an image address or a video address. "model-ver" refers to the version number of the image inference model, and "other-input-param" specifies which target configuration information from the image inference request needs to be read to form the query identifier; in this example, it's "appid" and "userid".

[0190] For example, if the configuration information includes the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model, then the query identifier can be composed of the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model.

[0191] S3. Obtain the image identification information of the image to be inferred and the version number of the image inference model from the image inference request based on the configuration information.

[0192] If the configuration information includes the image address, image digital signature, target configuration information, and version number of the image inference model, the electronic device can obtain the image identification information of the image to be inferred and the version number of the image inference model from the image inference request based on the configuration information. Thus, the image identification information of the image to be inferred and the version number of the image inference model can be combined to form a query identifier.

[0193] For example, the image address of the image to be inferred in the configuration information can be the image address "img-url", the target configuration information can include "appid" and "userid", and the version number of the image inference model is "model-ver". Then, based on this configuration information, the image address of the image to be inferred can be obtained from the image inference request as "img-url: https: / / xxxxx.com / xxx". The target configuration information can be "appid: QQQ" and "userid: ddd", and the version number of the image inference model can be "model-ver: 1". Then, the query identifier can be "img-url: https: / / xxxxx.com / xxx", "appid: QQQ", "userid: ddd", and "model-ver: 1".

[0194] In this embodiment, when the electronic device plays an image to be inferred, it triggers an image inference request to obtain the identification information of the image inference model currently used by the electronic device. Based on this identification information, it reads the pre-defined configuration information of the query identifier corresponding to the currently used image inference model to indicate the composition information of the query identifier. If the configuration information includes the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model, then it means that the query identifier can be composed of the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the image inference model. Based on the configuration information, it obtains the image identification information of the image to be inferred and the version number of the image inference model from the image inference request. This allows the electronic device to read the pre-configured configuration information based on the identifier of the image inference model, enabling it to determine the content category of the image to be inferred and the version of the image inference model from the information obtained from the image to be inferred and the currently used image inference model.

[0195] 205. The electronic device generates a query identifier corresponding to the image to be inferred based on the image identification information and the version number of the image inference model.

[0196] The image address can be the website address of the image or video in the image to be inferred. The target configuration information can be the identification information of the application software from which the image to be inferred originates and / or the user identity information of the user logging into the application software or electronic device, or the identity information of the user who logged into the electronic device to obtain the image to be inferred. The image inference model is used to perform image inference on the image to be inferred, thereby obtaining the inference result. The inference result can be the content category corresponding to the image to be inferred, or recommended content corresponding to the content category of the image to be inferred. The image inference model can be an image recognition model. The version number of the image inference model can represent the version of the currently used image inference model; if the version number is updated, it indicates that the version of the image inference model has been updated. The query identifier can be a cache key, and the query identifier can contain the image address of the image to be inferred, the image digital signature, the target configuration information, and the version number of the currently used image inference model. Based on the query identifier of the image to be inferred, a matching reference query identifier can be queried in the electronic device. If a matching reference query identifier exists, it indicates that the target inference result exists in the electronic device.

[0197] For example, an electronic device can generate a query identifier corresponding to an image to be inferred based on the image address, image digital signature, target configuration information, and the version number of the image inference model. This can include concatenating the image address, image digital signature, target configuration information, and the version number of the image inference model into the query identifier corresponding to the image to be inferred. If the obtained image address of the image to be inferred is "img-url: https: / / xxxxx.com / xxx", the target configuration information is "appid: QQQ" and "userid: ddd", and the version number of the image inference model is "model-ver: 1", then the query identifier corresponding to the image to be inferred could be "img-url: https: / / xxxxx.com / xxx", "appid: QQQ", "userid: ddd", and "model-ver: 1".

[0198] 206. The electronic device obtains a target inference result that matches the image to be inferred based on the query identifier. The target inference result is obtained from the reference inference result when the query identifier matches the reference query identifier. The target inference result is used to determine recommended data that matches the image to be inferred.

[0199] The query identifier is generated based on the image address, digital signature, target configuration information, and version number of the current image inference model in the electronic device. This query identifier can be associated with previously stored reference inference results on the electronic device. The target inference result is the inference result corresponding to the image to be inferred. This target inference result can be obtained from cached reference inference results on the electronic device or inferred from the image inference model. The output target inference result can be the content category of the image to be inferred output on the electronic device, or it can be recommended content corresponding to the content category of the image to be inferred, such as advertisements and bullet comments, displayed on the electronic device's display interface. If the target inference result is the content category or scene information of the image to be inferred, then the recommended data matched to the image to be inferred by the target inference result can be advertisement data or bullet comment data, etc.

[0200] For example, the reference inference result that matches the query identifier can be obtained from the reference inference result as the target inference result, or the target inference result can be obtained by performing image inference on the image to be inferred through the image inference model based on the query identifier, and the recommended data corresponding to the target inference result of the image to be inferred, such as advertisements and bullet comments, can be displayed on the display interface of the electronic device.

[0201] In one embodiment, obtaining the target inference result matching the image to be inferred based on the query identifier may include:

[0202] S1. Query the target reasoning result that matches the image to be reasoned from the image reasoning database according to the query identifier. The image reasoning database stores reference reasoning results and reference query identifiers. The reference reasoning results are obtained by performing image reasoning on the reference reasoning image. The reference query identifier is generated based on the image identification information of the reference reasoning image and the version number of the reference image reasoning model.

[0203] The query identifier refers to the query identifier corresponding to the image to be inferred. The image inference database can be set in an electronic device or in a cloud server connected to the electronic device. The image inference database can include associated stored reference inference results and reference query identifiers.

[0204] For example, an electronic device can compare a query identifier with a reference query identifier corresponding to a stored reference inference result to determine whether the reference query identifier corresponding to the reference inference result matches the query identifier corresponding to the image to be inferred. If they match, it means that a target inference result matching the image to be inferred has been found based on the query identifier.

[0205] S2. If a target reasoning result matching the image to be reasoned is found based on the query identifier, then the target reasoning result is output.

[0206] Optionally, if the query identifier corresponding to the image to be inferred is the same as the reference query identifier corresponding to the reference inference result, then the query identifier is determined to match the reference query identifier, that is, a target inference result matching the image to be inferred is found based on the query identifier. If a target inference result matching the image to be inferred is found based on the query identifier, it means that the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred are all the same as those of the reference image address, reference image digital signature, reference configuration information, and version number of the reference image inference model of the reference inference image.

[0207] For example, if the reference inference results cached by the electronic device include X, Y and Z, the reference query identifier corresponding to reference inference result X is L, the reference query identifier corresponding to reference inference result Y is M, and the reference query identifier corresponding to reference inference result Z is N, then if the query identifier corresponding to the image to be inferred is M, then the target inference result matching the image to be inferred is found according to the query identifier M, and this target inference result is the reference inference result Y.

[0208] For example, refer to Figure 3If the query identifier returns a result matching the image to be inferred, and the content category is "car," or recommended content corresponding to the car category, this recommended content could be a pre-stored link to a car for sale or an official website within the car category on the electronic device, or pre-stored pop-up comments related to the car category, such as "New model launched, up to x% off." (Refer to...) Figure 4 If the query identifier returns a result matching the image to be inferred, and the content category is a disease category, or recommended content corresponding to a disease category, this recommended content could be a pre-stored link to a cold medicine for sale or an official website for a disease category, or a pre-stored comment for a disease category, such as "Is AAA sick? I feel sorry for AAA." If a user clicks on the sales link based on the displayed recommended content, the user can be redirected to the sales interface of the recommended content.

[0209] Currently, online inference services for image models are characterized by numerous repetitive requests, high request processing latency, and high resource consumption. The image inference method provided in this embodiment can quickly query the corresponding target inference result when an image inference request is received, by using the corresponding query identifier. This allows the corresponding target inference result to be directly retrieved if it has been cached previously, without having to repeat the inference calculation, thus improving the efficiency of image inference.

[0210] In one embodiment, after retrieving a target inference result matching the image to be inferred from the image inference database based on a query identifier, the image inference method may further include:

[0211] S3. If no matching target reasoning result is found in the image reasoning database based on the query identifier, then extract the version number of the image reasoning model from the query identifier.

[0212] It should be noted that in this embodiment, if no target inference result matching the image to be inferred is found based on the query identifier, that is, if the query identifier of the image to be inferred is not the same as the reference query identifier cached in the electronic device, and since the query identifier is generated based on the image address, image digital signature, target configuration information, and version number of the image inference model of the image to be inferred, then if no target inference result matching the image to be inferred is found based on the query identifier, it means that at least one of the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred is different from at least one of the reference image address, reference image digital signature, reference configuration information, and version number of the reference image inference model of the reference inference image. Therefore, the target inference result cannot be directly obtained from the reference inference results cached in the electronic device, and it is necessary to perform image inference on the image to be inferred using the current image inference model to obtain the target inference result.

[0213] Optionally, if the query identifier corresponding to the image to be inferred is different from the reference query identifier corresponding to the reference inference result, it is determined that the query identifier and the reference query identifier do not match, that is, no target inference result matching the image to be inferred is found based on the query identifier.

[0214] For example, if the reference inference results cached by the electronic device include X, Y, and Z, and the reference query identifier corresponding to reference inference result X is L, the reference query identifier corresponding to reference inference result Y is M, and the reference query identifier corresponding to reference inference result Z is N, then if the query identifier corresponding to the image to be inferred is S, then no target inference result matching the image to be inferred is found according to the query identifier S. In this case, the version number of the image inference model is extracted from the query identifier so as to determine the image inference model currently used by the electronic device based on the version number, and the image inference model is used to perform image inference on the image to be inferred to obtain the target inference result.

[0215] S4. Perform image reasoning on the image to be reasoned using the image reasoning model corresponding to the version number, and obtain the target reasoning result.

[0216] Optionally, feature information of the image to be reasoned is extracted through an image reasoning model, the content of the image to be reasoned is identified based on the feature information, and image reasoning is performed on the image to be reasoned based on the content of the image to be reasoned to obtain the target reasoning result.

[0217] Optionally, after performing image reasoning on the image to be reasoned using the version number-corresponding image reasoning model to obtain the target reasoning result, the method may further include: associating and storing the target reasoning result with a query identifier. This allows the electronic device to directly retrieve the target reasoning result when it detects a new query identifier that matches the query identifier.

[0218] Reference Figure 5 As shown, Figure 5 This is an overall architecture diagram of the image inference system in this embodiment. The image inference system can be applied to electronic devices and may include a configuration center, a general caching service, an API gateway, an image inference model, a local image inference database, and object cloud storage. The image inference model is used to perform image inference on the acquired image to be inferred. The general caching service is used to cache reference inference results in the local image inference database or object cloud storage. When the image to be inferred is obtained through the API gateway, the target inference result is queried from the reference inference results based on the query identifier corresponding to the image to be inferred. The configuration center is used to configure the identification information corresponding to the current image inference model and the configuration information corresponding to the query identifiers of the cached content in the general caching service.

[0219] For example, an electronic device configures the identifier information of the currently used image inference model in the configuration center based on user operations, binds this identifier information to the currently used image inference model, and sets the configuration information of the currently used image inference model, thereby specifying the content contained in the query identifier. After the configuration information is set, the configuration center will synchronize the configuration information to the online inference general cache service of the image inference model in a hot update manner, which can realize the dynamic distribution of configuration information. It allows developers to update the configuration information without releasing a new version of the image inference model. Thus, when receiving an image to be inferred, the image identifier information in the image to be inferred and the version number of the currently used image inference model can be obtained according to the dynamic configuration information, thereby dynamically updating the query identifier, i.e., the cache key. It also allows developers to bypass the review mechanism in the electronic device, avoiding the costs caused by long review waits and multiple rejections, thereby reducing the development cost of the image inference device and improving the development efficiency of the image inference device. When an image to be inferred is played on an electronic device, an image inference request for that image is triggered. When the general caching service receives this request through the API gateway, it obtains the identifier information corresponding to the currently used image inference model. Based on this identifier, it reads the configuration information of the query identifier corresponding to that model. Using this configuration information, it retrieves the image identifier information of the image to be inferred from the request, as well as the version number of the image inference model. Based on the image identifier information and the version number, it generates a query identifier. Then, it searches the local image inference database for the target inference result based on this query identifier. If a matching reference query identifier is found in the local database, the reference inference result corresponding to that identifier is used as the target inference result. If no matching reference query identifier is found in the local database, the version number of the currently used image inference model is retrieved from the query identifier. Image inference is then performed on the image to be inferred using the model corresponding to that version number to obtain the target inference result. After obtaining the target inference result, it is associated with and stored with the query identifier.

[0220] In this embodiment, if no target inference result matching the image to be inferred is found based on the query identifier, it indicates that at least one of the image address, image digital signature, target configuration information, and version number of the currently used image inference model of the image to be inferred is different from at least one of the reference image address, reference image digital signature, reference configuration information, and version number of the reference image inference model of the reference inference image. Therefore, the target inference result cannot be directly obtained from the reference inference result cached by the electronic device. Instead, image inference of the image to be inferred needs to be performed using the current image inference model to obtain the target inference result. In other words, if the version number of the currently used image inference model is different from the version number of the reference image inference model, the reference inference result corresponding to the reference image inference model cannot be used. Only the currently used image inference model can be used to perform image inference of the image to be inferred to obtain the target inference result. Generally speaking, the currently used image inference model is an updated version compared to the reference image inference model. This ensures that after the new version of the image inference model is launched, all images to be inferred in the online system can use the new version of the image inference model for image inference calculation. This avoids the situation where some images or videos use the old version of the model for inference results, while some images or videos use the new version of the model for inference results, thus improving the image inference effect.

[0221] In this embodiment of the invention, the electronic device can obtain the image identification information of the image to be inferred, as well as the version number of the currently used image inference model; based on the image address, image digital signature, target configuration information, and the version number of the image inference model in the image identification information, a query identifier corresponding to the image to be inferred is generated; and the target inference result matching the image to be inferred is obtained based on the query identifier. Therefore, by obtaining the version number of the currently used image inference model each time an image inference request is received for subsequent image inference, the online system can use the new version of the image inference model for image inference calculations after a new version is released, thus avoiding the continued use of cached old version model inference results online after a model version update. Simultaneously, by using multiple information such as the image address, image digital signature, target configuration information, and image inference model version number of the image to be inferred to generate the query identifier, the target inference result can be obtained more accurately based on the query identifier, improving the accuracy of the image inference result.

[0222] To better implement the above methods, embodiments of the present invention also provide an image reasoning device, which can be integrated into an electronic device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer, and / or a personal computer.

[0223] For example, such as Figure 6As shown, the image reasoning device may include a first acquisition unit 301, a generation unit 302, and a second acquisition unit 303, as follows:

[0224] (1) The first acquisition unit 301 is used to acquire the image identification information of the image to be inferred and the version number of the currently used image inference model based on the image inference request when receiving the image inference request of the image to be inferred. The image inference model is used to make recommendations based on the image to be inferred.

[0225] For example, the first acquisition unit 301 can be used to acquire the identification information corresponding to the currently used image inference model when receiving an image inference request; read the configuration information of the query identifier corresponding to the image inference model according to the identification information; acquire the image identification information of the image to be inferred from the image inference request according to the configuration information; and acquire the version number of the image inference model.

[0226] Optionally, a storage unit 304 may be included before the first acquisition unit 301. The storage unit 304 can be used to acquire a reference inference image; perform image inference on the reference inference image through a reference image inference model to obtain a reference inference result; and associate and store the reference inference result with the reference query identifier corresponding to the reference inference result.

[0227] The storage unit 304 can also be used to upload the reference inference results to cloud storage; obtain the access address of the reference inference results in cloud storage; and associate the access address with the reference query identifier for storage.

[0228] Optionally, the image reasoning device may further include a reasoning unit 305, which is used to extract feature information of a reference reasoning image, the feature information including at least one of text features, sound features and image features; identify the scene of the reference reasoning image based on the feature information; and perform image reasoning on the reference reasoning image based on the scene of the reference reasoning image to obtain a reference reasoning result.

[0229] (2) Generation unit 302 is used to generate a query identifier corresponding to the image to be reasoned based on the image identification information and the version number of the image reasoning model.

[0230] For example, the image identifier information of the image to be inferred and the version number of the image inference model are concatenated to form the query identifier corresponding to the image to be inferred.

[0231] (3) The second acquisition unit 303 is used to acquire the target reasoning result that matches the image to be reasoned according to the query identifier. The target reasoning result is used to determine the recommended data that matches the image to be reasoned.

[0232] For example, the second acquisition unit 303 can be specifically used to query a target reasoning result that matches the image to be reasoned from an image reasoning database according to a query identifier. The image reasoning database stores reference reasoning results and reference query identifiers. The reference reasoning results are obtained by performing image reasoning on a reference reasoning image, and the reference query identifier is generated based on the image identification information of the reference reasoning image and the version number of the reference image reasoning model. If a target reasoning result that matches the image to be reasoned is found according to the query identifier, the target reasoning result is output. If no target reasoning result that matches the image to be reasoned is found according to the query identifier, the version number of the image reasoning model is extracted from the query identifier. Image reasoning is then performed on the image to be reasoned using the image reasoning model corresponding to the version number to obtain the target reasoning result.

[0233] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0234] As described above, in this embodiment, the first acquisition unit 301 is used to acquire the image identification information of the image to be inferred and the version number of the currently used image inference model; the generation unit 302 is used to generate a query identifier corresponding to the image to be inferred based on the image address, image digital signature, target configuration information, and the version number of the image inference model; and the second acquisition unit 303 is used to acquire the target inference result matching the image to be inferred based on the query identifier. Therefore, by acquiring the version number of the currently used image inference model each time an image inference request is received for subsequent image inference, the online system can use the new version of the image inference model for image inference calculations after a new version is released, thus avoiding the continued use of cached old version model inference results online after a model version update. Simultaneously, by using multiple information such as the image address, image digital signature, target configuration information, and image inference model version number of the image to be inferred to generate the query identifier, the target inference result can be acquired more accurately based on the query identifier, improving the accuracy of the image inference result.

[0235] This invention also provides an electronic device, such as... Figure 7 As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0236] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 7The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0237] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 402, and calls data stored in the memory 402, to perform various functions and process data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0238] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0239] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0240] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0241] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following computer program, and the processor 401 runs the application programs stored in the memory 402 to realize various functions, as follows:

[0242] The electronic device can obtain the image identification information of the image to be inferred, as well as the version number of the currently used image inference model; generate a query identifier corresponding to the image to be inferred based on the image address, image digital signature, target configuration information, and the version number of the image inference model; and obtain the target inference result that matches the image to be inferred based on the query identifier.

[0243] For example, an electronic device configures the identifier information of the currently used image inference model in the configuration center based on user operations, binds the identifier information to the currently used image inference model, and sets the configuration information of the currently used image inference model, thereby specifying the content of the query identifier. After the configuration information is set, the configuration center will synchronize the configuration information to the general cache service in a hot update manner. When the electronic device plays the image to be inferred, it triggers the image inference request for the image to be inferred. When the general cache service receives the image inference request for the image to be inferred through the API gateway, it obtains the identifier information corresponding to the currently used image inference model, reads the configuration information of the query identifier corresponding to the image inference model based on the identifier information, obtains the image identifier information of the image to be inferred and the version number of the image inference model from the image inference request based on the configuration information, generates a query identifier based on the image identifier information of the image to be inferred and the version number of the image inference model, and then searches for the target inference result in the local image inference database based on the query identifier. If a reference query identifier matching the query identifier is found in the local image inference database, the reference inference result corresponding to the reference query identifier is obtained as the target inference result. If no matching reference query identifier is found in the local image inference database, the version number of the currently used image inference model is obtained from the query identifier. Image inference is then performed on the image to be inferred using the image inference model corresponding to that version number to obtain the target inference result. After obtaining the target inference result, the target inference result is associated with and stored with the query identifier.

[0244] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0245] As described above, the electronic device in this embodiment of the invention can obtain the image identification information of the image to be inferred, as well as the version number of the currently used image inference model; generate a query identifier corresponding to the image to be inferred based on the image address, image digital signature, target configuration information, and the version number of the image inference model; and obtain the target inference result matching the image to be inferred based on the query identifier. Therefore, by obtaining the version number of the currently used image inference model each time an image inference request is received for subsequent image inference, the online system can use the new version of the image inference model for image inference calculations after a new version is released, thus avoiding the continued use of cached old version model inference results online after a model version update. Simultaneously, by using multiple pieces of information such as the image address, image digital signature, target configuration information, and image inference model version number of the image to be inferred to generate the query identifier, the target inference result can be obtained more accurately based on the query identifier, improving the accuracy of the image inference result.

[0246] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0247] To this end, embodiments of the present invention provide a computer-readable storage medium storing a plurality of computer programs, which can be loaded by a processor to execute steps in any of the image reasoning methods provided in the embodiments of the present invention. For example, the computer program can execute the following steps:

[0248] Obtain the image identifier information of the image to be inferred, and obtain the version number of the currently used image inference model; generate a query identifier corresponding to the image to be inferred based on the image address, image digital signature, target configuration information, and the version number of the image inference model; obtain the target inference result that matches the image to be inferred based on the query identifier.

[0249] For example, an electronic device configures the identifier information of the currently used image inference model in the configuration center based on user operations, binds the identifier information to the currently used image inference model, and sets the configuration information of the currently used image inference model, thereby specifying the content of the query identifier. After the configuration information is set, the configuration center will synchronize the configuration information to the general cache service in a hot update manner. When the electronic device plays the image to be inferred, it triggers the image inference request for the image to be inferred. When the general cache service receives the image inference request for the image to be inferred through the API gateway, it obtains the identifier information corresponding to the currently used image inference model, reads the configuration information of the query identifier corresponding to the image inference model based on the identifier information, obtains the image identifier information of the image to be inferred and the version number of the image inference model from the image inference request based on the configuration information, generates a query identifier based on the image identifier information of the image to be inferred and the version number of the image inference model, and then searches for the target inference result in the local image inference database based on the query identifier. If a reference query identifier matching the query identifier is found in the local image inference database, the reference inference result corresponding to the reference query identifier is obtained as the target inference result. If no matching reference query identifier is found in the local image inference database, the version number of the currently used image inference model is obtained from the query identifier. Image inference is then performed on the image to be inferred using the image inference model corresponding to that version number to obtain the target inference result. After obtaining the target inference result, the target inference result is associated with and stored with the query identifier.

[0250] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0251] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0252] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the image reasoning methods provided in the embodiments of the present invention, the beneficial effects that any of the image reasoning methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0253] According to one aspect of this application, a computer program product or computer program is provided, which is stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the methods provided in the various alternative implementations of the image reasoning aspect described above.

[0254] The foregoing has provided a detailed description of an image reasoning method, apparatus, and computer-readable storage medium provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image reasoning method, characterized in that, include: Obtain the reference inference image; Image reasoning is performed on the reference reasoning image using a reference image reasoning model to obtain the reference reasoning result; The reference inference result is associated and stored with the reference query identifier corresponding to the reference inference result; Upon receiving an image reasoning request for an image to be reasoned, the image identification information of the image to be reasoned and the version number of the currently used image reasoning model are obtained based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned. Based on the image identification information and the version number of the image reasoning model, a query identifier corresponding to the image to be reasoned is generated; The target inference result matching the image to be inferred is obtained according to the query identifier. The target inference result is used to determine the recommended data matching the image to be inferred. The target inference result is determined from the reference inference result. When the query identifier matches the reference query identifier, the target inference result matching the query identifier is determined from the reference inference result.

2. The method according to claim 1, characterized in that, The step of obtaining the target inference result matching the image to be inferred based on the query identifier includes: Extract the version number of the image reasoning model from the query identifier; The image reasoning model corresponding to the version number is used to perform image reasoning on the image to be reasoned, and the target reasoning result is obtained.

3. The method according to claim 1, characterized in that, The step of obtaining the target inference result matching the image to be inferred based on the query identifier includes: According to the query identifier, the target reasoning result matching the image to be reasoned is queried from the image reasoning database. The image reasoning database stores reference reasoning results and reference query identifiers. The reference reasoning result is obtained by performing image reasoning on the reference reasoning image. The reference query identifier is generated based on the image identification information of the reference reasoning image and the version number of the reference image reasoning model. If no target reasoning result matching the image to be reasoned is found according to the query identifier, then the version number of the image reasoning model is extracted from the query identifier; The image reasoning model corresponding to the version number is used to perform image reasoning on the image to be reasoned, and the target reasoning result is obtained.

4. The method according to claim 3, characterized in that, The step of retrieving a target reasoning result that matches the image to be reasoned from the image reasoning database based on the query identifier includes: If a target reasoning result matching the image to be reasoned is found based on the query identifier, then the target reasoning result is output.

5. The method according to claim 1, characterized in that, Upon receiving an image inference request for an image to be inferred, the step of obtaining the image identification information of the image to be inferred and the version number of the currently used image inference model based on the image inference request includes: Upon receiving an image reasoning request for an image to be reasoned, obtain the identification information corresponding to the currently used image reasoning model; The configuration information of the query identifier corresponding to the image inference model is read according to the identification information, wherein the configuration information is pre-set and synchronized to the image inference model through hot update; Based on the configuration information, obtain the image identification information of the image to be inferred from the image inference request, and obtain the version number of the currently used image inference model.

6. The method according to claim 1, characterized in that, The step of performing image reasoning on the reference inference image using a reference image reasoning model to obtain a reference inference result includes: Extract feature information from the reference inference image, wherein the feature information includes at least one of text features, sound features, and image features; Based on the feature information, identify the scene of the reference inference image; Based on the scene of the reference inference image, image inference is performed on the reference inference image to obtain the reference inference result.

7. The method according to claim 1, characterized in that, The step of associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result includes: The reference inference results are uploaded to cloud storage; Obtain the access address of the reference inference result in cloud storage; The access address is associated with and stored with the reference query identifier.

8. The method according to claim 7, characterized in that, The step of obtaining the target inference result matching the image to be inferred based on the query identifier includes: The access address of the target reasoning result matching the image to be reasoned is obtained in cloud storage based on the query identifier; The target reasoning result matching the image to be reasoned is obtained from cloud storage based on the access address.

9. The method according to claim 7, characterized in that, Uploading the reference inference result to cloud storage includes: Obtain the cached value of the reference inference result; If the cached value is greater than or equal to the preset value, the reference inference result is uploaded to cloud storage.

10. The method according to claim 9, characterized in that, The step of associating and storing the reference inference result with the reference query identifier corresponding to the reference inference result further includes: If the cached value is less than a preset value, the reference inference result is associated with and stored with the reference query identifier.

11. An image reasoning device, characterized in that, include: The first acquisition unit is used to acquire the reference inference image; Image reasoning is performed on the reference reasoning image using a reference image reasoning model to obtain the reference reasoning result; The reference inference result is associated and stored with the reference query identifier corresponding to the reference inference result; Upon receiving an image reasoning request for an image to be reasoned, the image identification information of the image to be reasoned and the version number of the currently used image reasoning model are obtained based on the image reasoning request. The image reasoning model is used to make recommendations based on the image to be reasoned. The generation unit is used to generate a query identifier corresponding to the image to be inferred based on the image identification information and the version number of the image inference model; The second acquisition unit is configured to acquire a target inference result matching the image to be inferred based on the query identifier. The target inference result is used to determine recommended data matching the image to be inferred. The target inference result is determined from the reference inference result. When the query identifier matches the reference query identifier, the target inference result matching the query identifier is determined from the reference inference result.

12. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the image reasoning method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to perform the image reasoning method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image reasoning method according to any one of claims 1 to 10.

Images