Data processing method, device and computer readable storage medium

Abstract

The present application discloses a data processing method, device and computer-readable storage medium, the method comprising: generating a target query feature corresponding to a query image through a target image recognition model; a target local feature having a similar association relationship with a historical query feature exists in the target query feature; based on the target material feature in the target image retrieval library, an associated target material image associated with the target query feature is obtained from the target image retrieval library; the target material feature refers to a feature generated by the target image recognition model; based on the historical material feature in the historical image retrieval library, an associated historical material image associated with the target local feature is obtained from the historical image retrieval library; the historical material feature refers to a feature generated by the historical image recognition model; and a recommended material image is determined in the associated historical material image and the associated target material image. The present application can reduce the consumption of computing resources and reduce the update time cost.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a data processing method, device, and computer-readable storage medium. Background Art

[0002] With the popularization of multimedia technology and the development of computer Internet, image data is growing at an alarming scale. The query object submits an image retrieval request, and the image retrieval library returns the material images or material videos that meet the requirements. This process can be called material retrieval.

[0003] How to find suitable materials for the query object in the massive image retrieval library conveniently and accurately has become a major research hotspot in the field of computer vision. In the prior art, in order to save storage space, the historical image retrieval library stores low-dimensional material features (such as 32-dimensional material features) corresponding to the historical material images. However, the larger the feature dimension used to represent the image information, the better the representation effect. Therefore, the query object will obtain the high-dimensional query features (such as 64-dimensional query features) of the query image to accurately describe the image retrieval requirements. Since the feature dimensions corresponding to the high-dimensional query features and the low-dimensional material features are inconsistent, it is impossible to perform image retrieval in the historical image retrieval library. It is necessary to perform feature expansion on the low-dimensional material features in the historical image database to obtain high-dimensional material features before image retrieval can be performed. However, the historical image material library often stores tens of millions or more historical material images. Therefore, feature expansion of low-dimensional material features, that is, updating low-dimensional material features, will consume a lot of computing resources and a lot of time costs. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, and computer-readable storage medium, which can reduce the consumption of computing resources and reduce the update time cost.

[0005] On the one hand, an embodiment of the present application provides a data processing method, including:

[0006] A query image is obtained, and a target query feature corresponding to the query image is generated by a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to a feature corresponding to the query image generated by the historical image recognition model;

[0007] Based on the target material features corresponding to the target material images in the target image retrieval library, the associated target material images associated with the target query features are obtained from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model;

[0008] Based on the historical material features corresponding to the historical material images in the historical image retrieval library, the associated historical material images associated with the target local features are obtained from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model;

[0009] Among the association history material images and the association target material images, a recommended material image for responding to the query image is determined.

[0010] On the one hand, an embodiment of the present application provides a data processing method, including:

[0011] Acquire a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image;

[0012] Generate a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image through a trained historical image recognition model;

[0013] Generate a target benchmark prediction feature corresponding to the benchmark sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image through an initial image recognition model; the model parameters in the initial image recognition model include the model parameters in the historical image recognition model;

[0014] According to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature, the model parameters in the initial image recognition model are adjusted to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; there are local features in the target query feature that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0015] An embodiment of the present application provides a data processing device, including:

[0016] The first acquisition module is used to acquire a query image and generate a target query feature corresponding to the query image through a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model;

[0017] The second acquisition module is used to acquire the associated target material image associated with the target query feature from the target image retrieval library based on the target material feature corresponding to the target material image in the target image retrieval library; the target material feature refers to the feature corresponding to the target material image generated by the target image recognition model;

[0018] The second acquisition module is further used to acquire, from the historical image retrieval library, associated historical material images associated with the target local features based on historical material features corresponding to the historical material images in the historical image retrieval library; the historical material features refer to features corresponding to the historical material images generated by the historical image recognition model;

[0019] The material determination module is used to determine a recommended material image for responding to the query image from the associated historical material images and the associated target material images.

[0020] Wherein, the first acquisition module includes:

[0021] A first input unit is used to input the query image into a target image recognition model; the target image recognition model includes a feature convolution layer, a feature pooling layer, and a target feature connection layer; the number of feature connection channels corresponding to the target feature connection layer is greater than the number of feature connection channels corresponding to the historical feature connection layer in the historical image recognition model;

[0022] A second input unit is used to perform feature convolution processing on the query image through the feature convolution layer, generate query features to be pooled corresponding to the query image, and input the query features to be pooled into the feature pooling layer;

[0023] A first generating unit is used to perform feature pooling processing on the query features to be pooled through the feature pooling layer, generate query features to be connected corresponding to the query image, and input the query features to be connected to the target feature connection layer;

[0024] The second generating unit is used to perform feature connection processing on the query feature to be connected through the target feature connection layer to generate the target query feature.

[0025] The data processing device further includes:

[0026] A third acquisition module is used to acquire a target material image and an image identifier corresponding to the target material image;

[0027] The first generation module is used to input the target material image into the target image recognition model, perform feature recognition processing on the target material image through the target image recognition model, and generate target material features;

[0028] The second generation module is used to use the image identifier corresponding to the target material image as the index key and the target material feature as the index value, generate an index table according to the index key and the index value, and store the index table in the target image retrieval library.

[0029] Wherein, the second acquisition module includes:

[0030] A first acquisition unit is used to acquire an index table in a target image retrieval library; the index table includes an index key and an index value, the index key is generated by an image identifier corresponding to the target material image, and the index value is generated by a target material feature;

[0031] A first determining unit is used to obtain a target query feature and a feature distance between index values, and determine an index value whose feature distance is less than a feature distance threshold as an associated index value;

[0032] The second determining unit is used to determine the index key corresponding to the associated index value as the associated index key, and obtain the associated target material image in the target image retrieval library according to the associated index key.

[0033] The target material image includes at least two target material images, and the target material feature includes target material features corresponding to the at least two target material images respectively;

[0034] The data processing device further comprises:

[0035] A fourth acquisition module, used to acquire at least two target material images, and input the at least two target material images into the target image recognition model respectively;

[0036] A third generating module is used to perform feature recognition processing on at least two target material images respectively through a target image recognition model to generate target material features corresponding to the at least two target material images respectively;

[0037] A fourth generating module is used to perform clustering processing on at least two target material features to generate A target feature clustering clusters and target cluster center features corresponding to the A target feature clustering clusters respectively; A is a positive integer greater than 1, and A is less than or equal to the total number of the at least two target material features;

[0038] The fourth generation module is further used to associate and store the A target feature clusters and the A target cluster center features in a target image retrieval library.

[0039] Among them, A target feature clusters include target feature clusters C d , d is a positive integer and d is less than or equal to A;

[0040] The fourth generation module includes:

[0041] The second acquisition unit is used to cluster the target features into clusters C d Get the target material feature E f , f is a positive integer, and f is less than or equal to the target feature cluster C d The total number of target material features in ;

[0042] The third acquisition unit is used to acquire the target material feature E f Corresponding target material image B f , get the target material image B f The corresponding image ID;

[0043] The third generating unit is used to generate the target material image B f The corresponding image identifier is used as the index key, and the target material feature E f As the index value, generate the cluster C with the target feature according to the index key and index value. d The associated index table;

[0044] The associated storage unit is used to associate and store each target cluster center feature and the index table corresponding to each target cluster center feature in the target image retrieval library.

[0045] Wherein, the second acquisition module includes:

[0046] The fourth acquisition unit is used to acquire A target cluster center features in the target image retrieval library; the A target cluster center features include target cluster center features I j , j is a positive integer and j is less than or equal to A;

[0047] The fifth acquisition unit is used to acquire the target query feature and the target cluster center feature I j The characteristic distance K between j ;

[0048] A third determining unit is used to sort the feature distances corresponding to the A target cluster center features, and determine the target feature distance from the sorted feature distances;

[0049] A sixth acquisition unit, configured to determine the target cluster center feature corresponding to the target feature distance as the associated target cluster center feature, and acquire an associated index table stored in association with the associated target cluster center feature;

[0050] The seventh acquisition unit is used to acquire the associated target material image associated with the target query feature according to the associated index table.

[0051] Among them, determine the material module, including:

[0052] A fourth determining unit is used to determine the associated historical material image and the associated target material image as candidate material images; the number of candidate material images is at least two;

[0053] an eighth acquisition unit, configured to acquire feature distances between the associated material features of each candidate material image and the target query feature;

[0054] a fifth determining unit, configured to sort at least two candidate material images according to the feature distance corresponding to each candidate material image;

[0055] The fifth determining unit is further configured to determine a recommended material image for responding to the query image from the at least two sorted candidate material images.

[0056] An embodiment of the present application provides a data processing device, including:

[0057] A sample acquisition module is used to acquire a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image;

[0058] A first generation module is used to generate a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image through a trained historical image recognition model;

[0059] A second generation module is used to generate a target reference prediction feature corresponding to the reference sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image through an image recognition initial model; the model parameters in the image recognition initial model include the model parameters in the historical image recognition model;

[0060] The third generation module is used to adjust the model parameters in the image recognition initial model according to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; there are local features in the target query feature that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0061] The sample acquisition module includes:

[0062] The first acquisition unit is used to acquire at least two positive sample pairs, and perform equal division processing on the at least two positive sample pairs to obtain T positive sample pair sets; T is a positive integer, and T is less than or equal to the total number of at least two positive sample pairs; the T positive sample pair sets include the positive sample pair set U v , v is a positive integer and v is less than or equal to T; the positive sample pair set U v The target positive sample pair includes a target reference sample image and a target positive sample image corresponding to the target reference sample image; the sample image set includes a positive sample pair set U v The sample image set O in p , p is a positive integer, and p is less than or equal to the set of positive sample pairs U v The total number of sample image sets in ;

[0063] The second acquisition unit is used to acquire the image distance between the target reference sample image and the remaining sample image; the remaining sample image refers to the remaining reference sample image in the remaining positive sample pair, or the remaining positive sample image in the remaining positive sample pair; the remaining positive sample pair includes the positive sample pair set U v Positive sample pairs other than the target positive sample pairs;

[0064] A first determining unit is used to sort the acquired image distances and determine a target remaining sample image from the remaining sample images according to the sorted image distances;

[0065] The second determining unit is used to determine the target residual sample image as the target negative sample image corresponding to the target reference sample image, and determine the target negative sample image and the target positive sample pair as the sample image set O. p .

[0066] Among them, the historical image recognition model includes a historical feature convolution layer, a historical feature pooling layer, and a historical feature connection layer;

[0067] The second generation module includes:

[0068] A third determining unit, used to determine the reference sample image, the positive sample image and the negative sample image as sample images to be processed;

[0069] The first input unit is used to input the sample image to be processed into the image recognition initial model; the image recognition initial model includes an initial feature convolution layer, an initial feature pooling layer and an initial target feature connection layer; wherein the parameters in the initial feature convolution layer are equal to the parameters in the historical feature convolution layer; the parameters in the initial feature pooling layer are equal to the parameters in the historical feature pooling layer; the initial target feature connection layer includes a historical feature connection channel and a target feature connection channel, and the parameters in the historical feature connection channel are equal to the parameters in the historical feature convolution layer;

[0070] The second input unit is used to perform feature convolution processing on the sample image to be processed through the initial feature convolution layer, generate prediction features of samples to be pooled corresponding to the sample image to be processed, and input the prediction features of samples to be pooled into the initial feature pooling layer;

[0071] The third input unit is used to perform feature pooling processing on the prediction features of the samples to be pooled through the initial feature pooling layer, generate prediction features of the samples to be connected corresponding to the sample images to be processed, and input the prediction features of the samples to be connected to the initial target feature connection layer;

[0072] A first processing unit is used to perform feature connection processing on the prediction features of the samples to be connected through the historical feature connection channel in the initial target feature connection layer to generate historical local prediction features;

[0073] A second processing unit is used to perform feature connection processing on the prediction features of the samples to be connected through the target feature connection channel in the initial target feature connection layer to generate target local prediction features;

[0074] The third processing unit is used to perform feature splicing processing on the historical local prediction features and the target local prediction features to obtain the target sample prediction features corresponding to the sample image to be processed; the target sample prediction features include target baseline prediction features, first target prediction features and second target prediction features.

[0075] The third generation module includes:

[0076] A third acquisition unit is used to acquire a first feature distance between the historical benchmark prediction feature and the first historical prediction feature, and to acquire a second feature distance between the historical benchmark prediction feature and the second historical prediction feature;

[0077] A fourth determining unit, configured to determine a historical model loss value corresponding to the historical image recognition model according to the first feature distance and the second feature distance;

[0078] a fourth acquisition unit, configured to acquire a third characteristic distance between the target reference prediction feature and the first target prediction feature, and to acquire a fourth characteristic distance between the target reference prediction feature and the second target prediction feature;

[0079] A fifth determining unit, configured to obtain a distance threshold generated based on the historical model loss value, and determine a total loss value corresponding to the image recognition initial model according to the third feature distance, the fourth feature distance, and the distance threshold;

[0080] The first generating unit is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer and the parameters in the target feature connection channel according to the total loss value to generate a target image recognition model.

[0081] The third generation module includes:

[0082] a sixth determining unit, configured to determine a historical model loss value corresponding to the historical image recognition model according to the historical benchmark prediction feature, the first historical prediction feature, and the second historical prediction feature;

[0083] The sixth determining unit is further used to determine a first loss value corresponding to the image recognition initial model according to the historical model loss value, the target benchmark prediction feature, the first target prediction feature, and the second target prediction feature;

[0084] a seventh determination unit, configured to obtain a historical local prediction feature, and determine a second loss value corresponding to the image recognition initial model according to the historical local prediction feature and the historical model loss value;

[0085] The seventh determining unit is further used to determine the sum of the first loss value and the second loss value as the total loss value of the image recognition initial model;

[0086] A statistical history unit is used to count the number of historical adjustments of the initial image recognition model; the number of historical adjustments is used to represent the number of times the model parameters in the initial image recognition model have been adjusted;

[0087] The second generation unit is used to adjust the model parameters in the initial image recognition model according to the total loss value and the number of historical adjustments to generate a target image recognition model.

[0088] The historical local prediction features include the historical reference local prediction features corresponding to the reference sample image, the first historical local prediction features corresponding to the positive sample image, and the second historical local prediction features corresponding to the negative sample image;

[0089] The seventh determination unit includes:

[0090] A first determining subunit, configured to determine the historical local prediction feature to be replaced from the historical benchmark local prediction feature, the first historical local prediction feature, and the second historical local prediction feature;

[0091] A second determination subunit is used for, when the historical local prediction feature to be replaced is the historical benchmark local prediction feature, obtaining a fifth feature distance between the historical benchmark prediction feature and the first historical local prediction feature, obtaining a sixth feature distance between the historical benchmark prediction feature and the second historical local prediction feature, and determining a second loss value corresponding to the image recognition initial model according to the fifth feature distance, the sixth feature distance and the historical model loss value;

[0092] The second determination subunit is further configured to, when the historical local prediction feature to be replaced is the first historical local prediction feature, obtain a seventh feature distance between the historical benchmark local prediction feature and the first historical prediction feature, obtain an eighth feature distance between the historical benchmark local prediction feature and the second historical local prediction feature, and determine a second loss value corresponding to the image recognition initial model according to the seventh feature distance, the eighth feature distance and the historical model loss value;

[0093] The second determination subunit is used to obtain the ninth feature distance between the historical benchmark local prediction feature and the first historical local prediction feature when the historical local prediction feature to be replaced is the second historical local prediction feature, and to obtain the tenth feature distance between the historical benchmark local prediction feature and the second historical prediction feature, and to determine the second loss value corresponding to the initial model of image recognition according to the ninth feature distance, the tenth feature distance and the historical model loss value.

[0094] Wherein, the second generation unit includes:

[0095] A first adjustment subunit is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel according to the total loss value when the number of historical adjustments does not meet the heartbeat update cycle, and obtain the target image recognition model based on the adjusted image recognition initial model;

[0096] The second adjustment subunit is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, the parameters in the target feature connection channel, and the parameters in the historical feature connection channel according to the total loss value when the number of historical adjustments meets the heartbeat update cycle, and obtain the target image recognition model based on the adjusted image recognition initial model.

[0097] On one hand, the present application provides a computer device, including: a processor, a memory, and a network interface;

[0098] The above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication function, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program so that the computer device executes the method in the embodiment of the present application.

[0099] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.

[0100] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the embodiment of the present application.

[0101] In an embodiment of the present application, a computer device can generate a target query feature corresponding to a query image through a target image recognition model, wherein the target image recognition model is trained based on a trained historical image recognition model, and it can be known that the model function of the target image recognition model may include the model function of the historical image recognition model, and the functional effect corresponding to the model function of the target image recognition model may be better than the functional effect of the model function of the historical image recognition model; since the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, it can be known that the image information recorded by the target query feature may be better than the image information recorded by the historical query feature; since there are target local features in the target query feature that have a similar association relationship with the historical query feature, it can be known that the target local features can replace the historical query features for image search processing; further, based on the target material feature, the computer device can obtain an associated target material image associated with the target query feature from a target image retrieval library; in addition, based on the historical material feature, an associated historical material image associated with the target local feature can be obtained from a historical image retrieval library; further, among the associated historical material images and the associated target material images, a recommended material image for responding to the query image can be determined. To sum up, the embodiments of the present application can not only generate target query features that record more image information, thereby improving the retrieval accuracy for the query image; it can also support the retrieval of related historical material images through target local features, so when there is a historical image retrieval library, it can avoid updating the historical material features in the historical material image library, thereby reducing the consumption of computing resources and reducing the update time cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0103] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0104] Figure 2 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0105] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0106] Figure 4 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0107] Figure 5 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0108] Figure 6 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0109] Figure 7 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0110] Figure 8 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0111] Fig. 9 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0112] Fig.10 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0113] Fig.11 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0114] Fig.12 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0115] Fig.13 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0116] Fig.14 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0117] Fig.15 is a structural schematic diagram of a computer device provided in an embodiment of the present application;

[0118] Fig.16It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0119] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0120] To facilitate understanding, some nouns are briefly explained as follows:

[0121] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0122] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0123] Computer vision technology (Computer Vision, CV) is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition. In the embodiment of the present application, computer vision technology can be used to identify object categories (such as people, dogs, cats, birds, etc.) in images.

[0124] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. In the embodiments of the present application, the target image recognition model and the historical image recognition model are both AI models based on machine learning technology, which can be used to identify and process images.

[0125] See also Figure 1 , Figure 1 Schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 As shown, the system may include a business server 100 and a terminal cluster. The terminal cluster may include: terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. It can be understood that the above system may include one or more terminal devices, and this application does not limit the number of terminal devices.

[0126] Among them, there can be a communication connection between terminal clusters, for example, there is a communication connection between terminal device 200a and terminal device 200b, and there is a communication connection between terminal device 200a and terminal device 200c. At the same time, any terminal device in the terminal cluster can have a communication connection with the business server 100, for example, there is a communication connection between terminal device 200a and business server 100, wherein the above communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, and this application does not limit it here.

[0127] It should be understood that Figure 1 Each terminal device in the terminal cluster shown in the figure may be installed with an application client. When the application client runs in each terminal device, it can be respectively connected to the above Figure 1 Data interaction is performed between the business servers 100 shown, that is, the above-mentioned communication connection. Among them, the application client can be an application client with image loading function such as a short video application, a video application, a live broadcast application, a social application, an instant messaging application, a game application, a music application, a shopping application, a novel application, a payment application, a browser, etc. Among them, the application client can be an independent client, or it can be an embedded sub-client integrated in a client (for example, a social client, an educational client, and a multimedia client, etc.), which is not limited here. Taking social applications as an example, the business server 100 can be a collection of multiple servers including a background server and a data processing server corresponding to the social application. Therefore, each terminal device can transmit data with the business server 100 through the application client corresponding to the social application. For example, each terminal device can upload its local image to the business server 100 through the application client of the social application, and then the business server 100 can send the image to other terminal devices or transmit it to the cloud server.

[0128] To facilitate subsequent understanding and description, the embodiments of the present application can be Figure 1A terminal device is selected as the target terminal device in the terminal cluster shown, for example, terminal device 200a is used as the target terminal device. When an image (such as image a) is acquired and an image retrieval instruction for image a is received, terminal device 200a can send image a as a query image to the business server 100. In the embodiment of the present application, the image to be searched is called a query image (such as the above-mentioned image a). Further, after the business server 100 receives the query image sent by the terminal device 200a, a target query feature corresponding to the query image can be generated by a target image recognition model, wherein the target image recognition model is trained based on a trained historical image recognition model, and there are target local features in the target query feature that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature. Therefore, the image feature representation effect corresponding to the target query feature in the embodiment of the present application is better than the image feature representation effect corresponding to the historical query feature, and the target local features in the target query feature can be used to replace the historical query feature for image search. The historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0129] Further, the business server 100 can obtain the associated target material image associated with the target query feature from the target image retrieval library based on the target material feature corresponding to the target material image in the target image retrieval library; wherein the target material feature refers to the feature corresponding to the target material image generated by the target image recognition model. Further, the business server 100 can obtain the associated historical material image associated with the target local feature from the historical image retrieval library based on the historical material feature corresponding to the historical material image in the historical image retrieval library; wherein the historical material feature refers to the feature corresponding to the historical material image generated by the historical image recognition model. In the embodiment of the present application, the image obtained after the search process is called the material image, and the target material image and the historical material image are both material images. The difference is that the upload timestamps corresponding to the target material image and the historical material image are different, and the upload timestamp corresponding to the historical material image is earlier than the upload timestamp corresponding to the target material image; the upload timestamp refers to the timestamp when the user uploads the image or video to the business server through the terminal device.

[0130] Further, the service server 100 may determine a recommended material image for responding to the query image from the associated historical material images and the associated target material images. Subsequently, the service server 100 may send the recommended material image to the terminal device 200a, and after the terminal device 200a receives the recommended material image sent by the service server 100, it may display the recommended material image on its corresponding screen.

[0131] Optionally, if the target image recognition model is stored locally on the terminal device 200a, the terminal device 200a can obtain the target query features corresponding to the query image through the target image recognition model, and then send the target query features to the business server 100. The process by which the business server 100 determines the recommended material image corresponding to the query image through the target query features is consistent with the process described above, so it is not repeated here. Among them, since training the target image recognition model involves a large amount of offline calculations, the target image recognition model local to the terminal device 200a can be sent to the terminal device 200a after training by the business server 100.

[0132] Optionally, if the terminal device 200a stores the above-mentioned target image recognition model, target image retrieval library and historical image retrieval library locally, the terminal device 200a can obtain the target query feature corresponding to the query image through the target image recognition model, and then obtain the associated target material image in the target image retrieval library based on the target query feature, and obtain the associated historical material image in the historical image retrieval library based on the target local feature. The subsequent process is consistent with the above description, so it is not repeated here. Among them, the target image recognition model, target image retrieval library and historical image retrieval library local to the terminal device 200a can all be sent to the terminal device 200a by the business server 100.

[0133] Optionally, it is understandable that the system architecture may include multiple business servers, a terminal device may be connected to a business server, each business server may obtain the query image uploaded by the terminal device connected thereto, thereby being able to identify the query image and obtain the recommended material image associated with the query image, and return the recommended material image to the terminal device connected thereto.

[0134] It should be noted that the above-mentioned business server 100, terminal device 200a, terminal device 200b, terminal device 200c..., terminal device 200n can all be blockchain nodes in the blockchain network, and the data described in the full text (such as query images and recommended material images) can be stored. The storage method can be that the blockchain node generates blocks according to the data and adds the blocks to the blockchain for storage.

[0135] Blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm. It is mainly used to organize data in chronological order and encrypt it into a ledger to make it impossible to be tampered with or forged. At the same time, it can verify, store and update data. Blockchain is essentially a decentralized database. Each node in the database stores the same blockchain. The blockchain network can divide nodes into core nodes, data nodes and light nodes. Core nodes, data nodes and light nodes together constitute blockchain nodes. Among them, the core node is responsible for the consensus of the entire blockchain network, that is, the core node is the consensus node in the blockchain network. The process of writing transaction data in the blockchain network into the ledger can be that the data node or light node in the blockchain network obtains the transaction data, and transmits the transaction data in the blockchain network (that is, the node transmits it in the form of a baton) until the consensus node receives the transaction data, and the consensus node then packs the transaction data into a block, executes consensus on the block, and writes the transaction data into the ledger after the consensus is completed. Here, the transaction data of query images and recommended material images are taken as examples. After reaching a consensus on the transaction data, the business server 100 (blockchain node) generates a block based on the transaction data and stores the block in the blockchain network. As for reading the transaction data (i.e., the query image and the recommended material image), the blockchain node can obtain the block containing the transaction data in the blockchain network, and further obtain the transaction data in the block.

[0136] It is understandable that the method provided in the embodiment of the present application can be performed by a computer device, and the computer device includes but is not limited to a terminal device or a business server. Among them, the business server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform. The terminal device can be a smart phone, tablet computer, laptop computer, desktop computer, PDA, mobile internet device (mobile internet device, MID), wearable device (such as smart watch, smart bracelet, etc.), smart computer, smart car, etc., which can run the above-mentioned application. Intelligent terminal. Among them, the terminal device and the business server can be directly or indirectly connected by wired or wireless means, and the embodiment of the present application is not limited here.

[0137] It can be understood that the above system architecture can be applied to business scenarios such as image recommendation scenarios, image distribution scenarios, image search scenarios, etc., and specific business scenarios will not be listed one by one here.

[0138] For further information, see Figure 2 , Figure 2 1 is a schematic diagram of a data processing scenario provided by an embodiment of the present application. The implementation process of the data processing scenario can be performed in a business server, in a terminal device, or interactively in a terminal device and a business server, which is not limited here. The terminal device can be the above-mentioned Figure 1 Any terminal device in the terminal cluster of the corresponding embodiment, Figure 2 Taking the terminal device 200a as an example, the service server can be the above Figure 1 The corresponding business server 100 of the embodiment.

[0139] like Figure 2 As shown, user 201a can enter the query image (i.e. Figure 2 The image 202a in the image is uploaded to an application client (e.g., a browser) with an image search function on the terminal device 200a. Optionally, the application client may be integrated with a camera function, so the image 202a may be an image captured by the camera function on the application client, for example, the image 202a is obtained by photographing a cupcake. Optionally, the image 202a may also be a cupcake image obtained by the user 201a from other platforms. The embodiment of the present application does not limit the specific content and source of the query image. Further, when the user 201a clicks the search control 203a on the application client, the terminal device 200a may respond to the search operation and send the image 202a to the business server 100.

[0140] Further, after the business server 100 obtains the query image (i.e., image 202a) sent by the terminal device 200a, the image 202a can be input into the target image recognition model 100a, and the target image recognition model 100a performs image recognition processing on the image 202a, and extracts the image features of the image 202a, and generates the target query feature 100b corresponding to the image 202a. The target image recognition model 100a in the embodiment of the present application is trained based on the trained historical image recognition model 100c, so the model function of the target image recognition model 100a includes but is not limited to the model function of the historical image recognition model 100c, and the functional effect corresponding to the model function of the target image recognition model 100a is better than the unique functional effect of the model function of the historical image recognition model 100c, and the above-mentioned model function includes but is not limited to the image information representation effect of the query feature (such as the target query feature 100b), which is better than the image information representation effect of the historical feature (such as the historical query feature).

[0141] Please see again Figure 2, the target query feature 100b may include a target local feature 1001b, the feature dimension corresponding to the target local feature 1001b is equal to the feature dimension corresponding to the historical feature (such as the historical query feature) generated by the historical image recognition model 100c, and the target local feature 1001b has a similar correlation with the historical query feature, so after obtaining the target local feature 1001b, the business server 100 does not need to generate the historical query feature corresponding to the query image through the historical image recognition model 100c, and can directly replace the historical query feature with the target local feature 1001b. It can be understood that the feature dimension corresponding to the target query feature 100b is greater than the feature dimension corresponding to the historical query feature, so the representation effect of the target query feature 100b on the image information of the query image is better than the representation effect of the historical query feature on the image information of the query image, so the target query feature 100b can improve the retrieval accuracy for the query image.

[0142] Further, the service server 100 obtains the target image retrieval library 100e, such as Figure 2 As shown, the target image retrieval library 100e may include a target material image and a target material feature corresponding to the target material image. The target image retrieval library 100e may include one or more target material images. The embodiment of the present application does not limit the number of target material images. Among them, the target material feature refers to the feature corresponding to the target material image generated by the target image recognition model 100a. Figure 2 In the example of target material images, image 1001f, image 1002f, ..., image 100nf are used, and the target material features may include feature 1001e corresponding to image 1001f generated by target image recognition model 100a, feature 1002e corresponding to image 1002f generated by target image recognition model 100a, ..., feature 100ne corresponding to image 100nf generated by target image recognition model 100a, where n is a positive integer. Further, the business server 100 may obtain the associated target material images from the target image retrieval library 100e according to the feature distances respectively corresponding to the target query feature 100e and each target material feature, such as Figure 2 In the example, images 1002f, ..., and image 100nf are determined as associated target material images. For a description of the feature distance between the target query feature and the target material feature, and a description of determining the associated target material image based on the feature distance, please refer to the following. Figure 3It is understandable that, in addition to storing the target material image and the target material features, the target image retrieval library 100e may also store other data, such as the image identifier corresponding to the target material image. The data contained in the target image retrieval library 100e is not limited here, and data can be stored according to the actual application scenario.

[0143] Further, the service server 100 obtains the historical image retrieval library 100h, such as Figure 2 As shown, the historical image retrieval library 100h may include historical material images and historical material features corresponding to the historical material images. The historical image retrieval library 100h may include one or more historical material images. The embodiment of the present application does not limit the number of historical material images. Among them, the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model 100c. Figure 2 In , taking image 1001i, image 1002i, ..., image 100mi as examples of historical material images, the historical material features may include feature 1001h corresponding to image 1001i generated by historical image recognition model 100c, feature 1002h corresponding to image 1002i generated by historical image recognition model 100c, ..., feature 100mh corresponding to image 100mi generated by historical image recognition model 100c, where m is a positive integer. Further, the business server 100 may obtain the associated historical material images from the historical image retrieval library 100h according to the feature distances respectively corresponding to the target local feature 1001b and each historical material feature, such as Figure 2 In the example, images 1001i, ..., and image 1002i are determined as associated historical material images. For a description of the feature distance between the target query feature and the historical material feature, and a description of determining the associated historical material image based on the feature distance, please refer to the following Figure 3 It is understandable that, in addition to storing historical material images and historical material features, the historical image retrieval library 100h may also store other data, such as image identifiers corresponding to historical material images. The data contained in the historical image retrieval library 100h is not limited here, and data can be stored according to actual application scenarios.

[0144] Furthermore, the service server 100 comprehensively considers two image retrieval libraries (i.e. Figure 2 The target image retrieval library 100e and the historical image retrieval library 100h) in the associated material image, the associated material image includes Figure 2The image 1001i, image 1002i, ..., image 1002f and image 100nf in the example can determine the recommended material image for responding to the query image according to the image distances respectively corresponding to each associated material image and the query image. Figure 2 As shown, the recommended material images may include images 1002i, 1002f, and 100nf, wherein image 1002i is from the historical image retrieval library 100h, and images 1002f and 100nf are both from the target image retrieval library 100e. For a description of the image distance between the associated material image and the query image, and a description of determining the recommended material image based on the image distance, please refer to the following Figure 3 The description in the corresponding embodiment will not be expanded here.

[0145] Further, the service server 100 returns the recommended material image to the terminal device 200a, such as Figure 2 As shown, after the terminal device 200a obtains the recommended material image returned by the service server 100, the user 201a can view the recommended material image through the display screen corresponding to the terminal device 200a.

[0146] Understandably, Figure 2 The interfaces and controls shown are only some forms of expression for reference. In actual business scenarios, developers can make relevant designs according to product requirements. The embodiments of the present application do not limit the specific forms of the interfaces and controls involved.

[0147] From the above, it can be seen that the embodiments of the present application can not only obtain associated target material images associated with target material features from the target image retrieval library, but also obtain associated historical material images associated with target local features from the historical image retrieval library, that is, in the scenario where there are dual image retrieval libraries (that is, the target image retrieval library and the historical image retrieval library), and the target material features (belonging to high-dimensional material features) and the historical material features (belonging to low-dimensional material features) respectively correspond to different feature dimensions, the embodiments of the present application do not need to update the historical material features in the historical image retrieval library, and can perform image retrieval in the historical image retrieval library according to the target local features in the target query features, so that the consumption of computing resources and the update time cost can be reduced; in addition, through the target query features (belonging to high-dimensional query features), the image retrieval requirements can be accurately expressed, thereby improving the retrieval accuracy for the query image.

[0148] For further information, see Figure 3 , Figure 3 is a flow chart of a data processing method provided by an embodiment of the present application. The data processing method can be performed by a business server (for example, the above Figure 1The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Figure 3 As shown, the data processing method may at least include the following steps S101 to S104.

[0149] Step S101, obtain a query image, and generate a target query feature corresponding to the query image through a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0150] For a trained model, tasks such as updating only some network parameters or adding new sample data to retrain network parameters are all fine-tuning learning, and the purpose of fine-tuning is to obtain a better model based on the original model parameters. In an embodiment of the present application, the historical image recognition model can be the original model, and the target image recognition model can be a better model obtained after fine-tuning the original model (i.e., the historical image recognition model).

[0151] Feature expansion refers to the increase in the dimension of the feature vector, such as the dimension of the feature vector (embedding) increases from 32 to 64. In AI models, the larger the dimension of the vector used to represent image information, the better the feature representation effect. On the other hand, large-dimensional feature vectors require more storage space.

[0152] The present application embodiment is described based on fine-tuning learning and feature expansion. For details, please refer to Figure 4 , Figure 4 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 4As shown, the business server obtains the query image 30a, and inputs the query image 30a into the target image recognition model 30b, and generates the target query feature 30c corresponding to the query image 30a through the target image recognition model 30b. Among them, the target image recognition model 30b is trained based on the trained historical image recognition model 30f. In the embodiment of the present application, the model used for image recognition processing is called an image recognition model. The target image recognition model 30b and the historical image recognition model 30f are both image recognition models. The model structures corresponding to the target image recognition model 30b and the historical image recognition model 30f are consistent. The difference is that the number of feature connection channels of the target feature connection layer contained in the target image recognition model 30b is greater than the number of feature connection channels of the feature connection layer contained in the historical image recognition model 30f. Therefore, the feature dimensions corresponding to the target features (including the above-mentioned target query features 30c and target material features) obtained by the target image recognition model 30b are all greater than the feature dimensions corresponding to the historical features (including historical query features and historical material features) obtained by the historical image recognition model 30f.

[0153] It should be noted that the embodiment of the present application does not limit the model type of the image recognition model (including the target image recognition model 30b and the historical image recognition model 30f), and can be any model that can perform image recognition processing, that is, it can be set according to the actual application scenario. In addition, the embodiment of the present application does not limit the model structure of the image recognition model, and the model structure can be set according to the actual application scenario.

[0154] In the embodiment of the present application, the feature connection channels included in the target image recognition model 30b are divided into historical feature connection channels and target feature connection channels, wherein the historical feature connection channels correspond to the (overall) feature connection channels in the historical image recognition model 30f, and since the target image recognition model 30b is obtained by training based on the historical image recognition model 30f, the target local features 301c generated by the historical feature connection channels have a similar correlation relationship with the historical query features, that is, the feature distances corresponding to the two features are close. In the embodiment of the present application, the feature connection channel used for feature dimension expansion is called the target feature connection channel, that is, the dimension difference between the feature dimension corresponding to the target feature and the feature dimension corresponding to the historical feature is equal to the number of target feature connection channels. In the embodiment of the present application, the number of historical feature connection channels and the number of target feature connection channels are not limited, and in actual application, they can be set according to the scene.

[0155] It should be noted that the embodiment of the present application does not set the channel order of the historical feature connection channel and the target feature connection channel. It only needs to be that the historical feature connection channel is a continuous channel. Optionally, the historical feature connection channel can be located before the target feature connection channel. In this case, the target local feature is the leading dimension (equal to the number of historical feature connection channels) feature in the target query feature; optionally, the historical feature connection channel can be located after the target feature connection channel. In this case, the target local feature is the trailing dimension (equal to the number of historical feature connection channels) feature in the target query feature; optionally, the historical feature connection channel can be located in the middle of the target feature connection channel. In this case, the target local feature is the middle dimension (equal to the number of historical feature connection channels) feature in the target query feature.

[0156] For ease of understanding and description, the embodiments of the present application are described in Figure 4 The number of feature connection channels of the target feature connection layer of the target image recognition model 30b is set to 64 (i.e. Figure 2 64 channels in the historical image recognition model 30f), and the number of feature connection channels of the feature connection layer is set to 32 (i.e. Figure 2 Similarly, for ease of understanding and description, Figure 4 The first 32 channels of the target feature connection layer of the target image recognition model 30b are set to the historical feature connection channels corresponding to the feature connection layers of the historical image recognition model 30f. At this time, the feature dimension of the target local feature 301c is 32 dimensions, the feature dimension of the target query feature 30c is 64 dimensions, and the target local feature 301c is the first 32-dimensional feature of the target query feature 30c. At the same time, the last 32-dimensional feature of the target query feature 30c is the feature generated by the target feature connection channel.

[0157] It can be understood that the target query feature 30c is a high-dimensional semantic vector used to characterize the query image 30a, while the historical query feature is a low-dimensional semantic vector used to characterize the query image 30a, so the image information characterization effect of the target query feature 30c is stronger than that of the historical query feature.

[0158] Step S102, based on the target material features corresponding to the target material images in the target image retrieval library, obtain the associated target material images associated with the target query features from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model.

[0159] Specifically, the target image retrieval library in the embodiment of the present application can be regarded as an electronic file cabinet - a place to store electronic files (this application may include target material images and target material features, etc.), and the business server can add, query, update, delete, etc. the data in the file. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application.

[0160] Please combine Figure 4 As for step S101, it can be seen that the target material features corresponding to the target material image generated by the target image recognition model 30b are all 64-dimensional, which is equivalent to Figure 4 Therefore, the target material features (64-dimensional material features) in the target image retrieval library 30e are all high-dimensional material features compared to the historical material features (32-dimensional material features).

[0161] After generating the target query feature 30c, the business server can perform similarity calculation on the target query feature 30c and the target material feature to obtain the similarity corresponding to the two features. It can be understood that the greater the similarity between the target query feature 30c and the target material feature, the smaller the feature distance between the target query feature 30c and the target material feature, which can further indicate that the image distance between the query image and the target material image is small, that is, the query image and the target material image can be similar images or the same image; conversely, the smaller the similarity between the target query feature 30c and the target material feature, the greater the feature distance between the target query feature 30c and the target material feature, which can further indicate that the image distance between the query image and the target material image is large, that is, the query image and the target material image are different images.

[0162] Further, the business server may determine a target material image whose feature distance is less than a feature distance threshold as an associated target material image, for example, determining a target material feature whose feature distance is less than 0.2 (feature distance threshold) as an associated target material image from the target image retrieval library 30e; or, determining a target material image whose similarity is greater than a similarity threshold as an associated target material image; or, obtaining associated target material images that match a predetermined number from the target material images in order of feature distance from small to large, for example, determining target material images whose feature distances are ranked in the top ten from the target image retrieval library 30e as associated target material images.

[0163] It is understandable that the embodiment of the present application does not limit the method of retrieving the associated target material image from the target image retrieval library 30e. It can be a brute force search method (suitable for the scenario where the number of target material features in the target image retrieval library 30e is not large, such as 1,000 target material features); it can also be a brute force search optimization method (suitable for the scenario where the number of target material features in the target image retrieval library 30e is large, such as hundreds of millions of target material features), such as an image bucket retrieval method, where one bucket corresponds to one cluster. For a description of the above two methods, please see below. Figure 5 as well as Figure 7 The descriptions in the corresponding embodiments are not expanded here.

[0164] It is understandable that the embodiment of the present application does not limit the method of obtaining the feature distance between the target query feature 30c and the target material feature. Algorithms such as Euclidean distance, cosine similarity, and Jaccard similarity can be used, and the method for obtaining the feature distance can be set according to the actual application scenario.

[0165] Step S103, based on the historical material features corresponding to the historical material images in the historical image retrieval library, obtain the associated historical material images associated with the target local features from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model.

[0166] Specifically, the historical image retrieval library in the embodiment of the present application can be regarded as another electronic file cabinet - a place for storing electronic files (the present application may include historical material images and historical material features, etc.). It can be understood that the historical image retrieval library and the target image retrieval library are both image retrieval libraries. The difference between the two is that the library creation timestamp is different. The library creation timestamp of the historical image retrieval library is earlier than the library creation timestamp of the target image retrieval library, but the two image retrieval libraries can exist at the same time, and the historical image retrieval library can be eliminated before the target image retrieval library, that is, the library is destroyed. In the scenario where there is only a historical image retrieval library, the user uploads images or videos to the business server through the terminal device. At this time, the business server stores the media data uploaded by the user in the historical image retrieval library and uses it as a historical material image; when there are two image retrieval libraries, the media data uploaded by the user (including images and videos) will be stored in the target image retrieval library to generate a target material image.

[0167] Please combine Figure 4 As for step S101, it can be seen that the historical material features corresponding to the historical material images generated by the historical image recognition model 30f are all 32-dimensional, which is equivalent to Figure 4Therefore, the historical material features (32-dimensional material features) in the historical image retrieval library 30d are all low-dimensional material features compared to the target material features (64-dimensional material features).

[0168] After generating the target query feature 30c, the business server can obtain the target local feature 301c having a similar association relationship with the historical query feature, and further, can perform similarity calculation on the target local feature 301c and the historical material feature to obtain the similarity corresponding to the two features. It can be understood that the greater the similarity between the target local feature 301c and the historical material feature, the smaller the feature distance between the target local feature 301c and the historical material feature, and further, it can be explained that the image distance between the query image and the historical material image is small, that is, the query image and the historical material image can be similar images or the same image; conversely, the smaller the similarity between the target local feature 301c and the historical material feature, the greater the feature distance between the target local feature 301c and the historical material feature, and further, it can be explained that the image distance between the query image and the historical material image is large, that is, the query image and the historical material image are different images.

[0169] Furthermore, the business server may determine historical material images whose feature distance is less than a feature distance threshold as associated historical material images, for example, determining historical material features whose feature distance is less than 0.3 (feature distance threshold) from the historical image retrieval library 30d as associated historical material images; or, determining historical material images whose similarity is greater than a similarity threshold as associated historical material images; or, obtaining associated historical material images that match a predetermined number from historical material images in order of feature distance from small to large, for example, determining historical material images whose feature distance ranks in the top 20 from the historical image retrieval library 30d as associated historical material images.

[0170] It can be understood that, as described in step S102, the embodiment of the present application does not limit the method of retrieving the associated historical material image from the historical image retrieval library 30d, nor does it limit the method of the feature distance between the target local feature 301c and the historical material feature.

[0171] Step S104: determining a recommended material image for responding to the query image from the associated historical material images and the associated target material images.

[0172] Specifically, the embodiment of the present application adopts a dual image retrieval library (including a historical image retrieval library and a target image retrieval library) to search for recommended material images for query images. Therefore, after obtaining the retrieval results returned by the two indexing systems (including associated target material images and associated historical material images), the retrieval results can be merged first, and all the returned retrieval results can be sorted from small to large according to the feature distances corresponding to the retrieval results, and the recommended material images for the query image can be determined according to the sorting results.

[0173] Optionally, the business server may determine the recommended material image for responding to the query image based on the image distance and the selection weights corresponding to the associated historical material image and the associated target material image. For example, if the business server gives priority to the associated target material image, the selection weight corresponding to the associated target material image may be set to a coefficient a1 less than 0.5, and the selection weight corresponding to the associated historical material image may be set to a coefficient a2 greater than 0.5. At this time, the business server determines the recommended material image for responding to the query image from the associated historical material image and the associated target material image based on the image distance and the selection coefficient. For example, the business server sums the image distance corresponding to the associated historical material image and the selection weight corresponding to the associated historical material image to obtain the historical recommendation result for the associated historical material image, sums the image distance corresponding to the associated target material image and the selection weight corresponding to the associated target material image to obtain the target recommendation result for the associated target material image, compares the target recommendation result with the historical recommendation result, and if the target recommendation result is less than the historical recommendation result, selects the associated target material image; if the target recommendation result is equal to or greater than the historical recommendation result, selects the associated historical material image.

[0174] Optionally, if the associated target material image is preferred, the business server may set the selection weight corresponding to the associated target material image to a coefficient a3 less than 1, and set the selection weight corresponding to the associated historical material image to a coefficient a4 greater than 1. At this time, the business server determines the recommended material image for responding to the query image from the associated historical material image and the associated target material image based on the image distance and the selection coefficient. For example, the business server multiplies the image distance corresponding to the associated historical material image by the selection weight corresponding to the associated historical material image to obtain the historical recommendation result for the associated historical material image, and multiplies the image distance corresponding to the associated target material image by the selection weight corresponding to the associated target material image to obtain the target recommendation result for the associated target material image. Further, the business server compares the target recommendation result with the historical recommendation result. If the target recommendation result is less than the historical recommendation result, the associated target material image is selected. If the target recommendation result is equal to or greater than the historical recommendation result, the associated historical material image is selected. Optionally, the business server may also perform weighted summation processing on the image distance corresponding to the associated historical material image and the selection weight corresponding to the associated historical material image to obtain a historical recommendation result for the associated historical material image, perform weighted summation processing on the image distance corresponding to the associated target material image and the selection weight corresponding to the associated target material image to obtain a target recommendation result for the associated target material image, and then compare the historical recommendation result with the target recommendation result. The subsequent process is the same as above, so it will not be repeated.

[0175] In an embodiment of the present application, a computer device can generate a target query feature corresponding to a query image through a target image recognition model, wherein the target image recognition model is trained based on a trained historical image recognition model, and it can be known that the model function of the target image recognition model may include the model function of the historical image recognition model, and the functional effect corresponding to the model function of the target image recognition model may be better than the functional effect of the model function of the historical image recognition model; since the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, it can be known that the image information recorded by the target query feature may be better than the image information recorded by the historical query feature; since there are target local features in the target query feature that have a similar association relationship with the historical query feature, it can be known that the target local features can replace the historical query features for image search processing; further, based on the target material feature, the computer device can obtain an associated target material image associated with the target query feature from a target image retrieval library; in addition, based on the historical material feature, an associated historical material image associated with the target local feature can be obtained from a historical image retrieval library; further, among the associated historical material images and the associated target material images, a recommended material image for responding to the query image can be determined. To sum up, the embodiments of the present application can not only generate target query features that record more image information, thereby improving the retrieval accuracy for the query image; it can also support the retrieval of related historical material images through target local features, so when there is a historical image retrieval library, it can avoid updating the historical material features in the historical material image library, thereby reducing the consumption of computing resources and reducing the update time cost.

[0176] See also Figure 5 , Figure 5 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Figure 5 As shown, the method may include at least the following steps.

[0177] Step S201, obtaining a target material image, and obtaining an image identifier corresponding to the target material image.

[0178] Specifically, the business server needs to establish an index relationship for the image before it can use the image as a material image, and then the image can be matched when the user searches for an image. Figure 6 , Figure 6Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 6 As shown, the service server obtains the target material image 50a, and further obtains the image identifier 501a corresponding to the target material image 50a, such as Figure 5 The illustrated identifier is Jhhgggnhiiiiu142. It is understandable that the image identifier can be any information that can be used to identify the target material image 50a. For example, the image identifier is a hash value generated by the service server performing a hash operation on the material name or storage address of the target material image 50a through a hash algorithm.

[0179] Step S202: input the target material image into a target image recognition model, perform feature recognition processing on the target material image through the target image recognition model, and generate target material features.

[0180] Specifically, the specific process of step S202 can refer to the description of step S204 below. The difference between the two steps is that the images input into the target image recognition model are different. In this step, the target material image is input into the target image recognition model. Therefore, in this step, the target material features corresponding to the target material image are generated, such as Figure 6 The target material feature 50c shown in is a 64-dimensional feature vector; step S204 inputs the query image into the target image recognition model, thereby generating a target query feature corresponding to the query image, which is another 64-dimensional feature vector, so it is not repeated here.

[0181] Step S203: using the image identifier corresponding to the target material image as an index key and the target material feature as an index value, generating an index table according to the index key and the index value, and storing the index table in the target image retrieval library.

[0182] For details, please refer to Figure 6 The service server identifies the image corresponding to the target material image 50a (eg Figure 6 The identifier 501a in the target image retrieval library 50e is used as the index key, and the target material feature 50c is used as the index value, so an index key-value pair including the index key and the index value can be generated. Further, the generated index key-value pair can be stored in the index table 501e in the target image retrieval library 50e. It can be understood that the embodiment of the present application only takes the target material image 50a as an example of the process of establishing an index relationship for the image. The process of the business server establishing an index relationship for the image uploaded by other users and determining it as the target material image can refer to the above process.

[0183] The embodiment of the present application does not update the historical material features in the historical image material library 50d.

[0184] Step S204, obtain a query image, and generate a target query feature corresponding to the query image through a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0185] Specifically, the business server inputs the query image into the target image recognition model; the target image recognition model includes a feature convolution layer, a feature pooling layer and a target feature connection layer; the number of feature connection channels corresponding to the target feature connection layer is greater than the number of feature connection channels corresponding to the historical feature connection layer in the historical image recognition model; the query image is subjected to feature convolution processing through the feature convolution layer to generate query features to be pooled corresponding to the query image, and the query features to be pooled are input into the feature pooling layer; the query features to be pooled are subjected to feature pooling processing through the feature pooling layer to generate query features to be connected corresponding to the query image, and the query features to be connected are input into the target feature connection layer; the query features to be connected are subjected to feature connection processing through the target feature connection layer to generate target query features.

[0186] In the embodiment of the present application, the model types corresponding to the target image recognition model and the historical image recognition model are not limited, and can be composed of any one or more neural network models, such as conditional random fields (CRFs), convolutional neural networks (CNN), fully convolutional networks (FCNs), etc. Similarly, the embodiment of the present application does not limit the model structures corresponding to the target image recognition model and the historical image recognition model, and the model structure can be set according to the actual application scenario.

[0187] For ease of understanding, the present application embodiment is described with the target image recognition model and the historical image recognition model as examples of residual networks. Further, please refer to Table 1 and Table 2. Table 1 is a structural schematic table of a historical image recognition model provided in the present application embodiment, and Table 2 is a structural schematic diagram of a target image recognition model provided in the present application embodiment. Among them, Table 1 and Table 2 are both the network structures of the residual network ResNet-101. The difference in the model structures corresponding to the target image recognition model and the historical image recognition model is that the feature connection channel (32 channels) corresponding to the fully connected layer 1 (Fully Connected Layer, FC1, equal to the above-mentioned feature connection layer) in Table 1 is smaller than the feature connection channel (64 channels) corresponding to the fully connected layer 2 (Fully Connected Layer, FC2, equal to the above-mentioned target feature connection layer) in Table 2, so the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature. Here, only a brief explanation of each network layer in Table 1 is given, and the meaning of each layer in Table 2 can refer to the following description.

[0188] Table 1 may include convolutional layers (Layer name), output image size (Output size) and convolution information in each convolutional layer. As shown in Table 1, the historical image recognition model may include 7 convolutional network layers, specifically including convolutional network layer 1 (e.g., Conv1), convolutional network layer 2 (e.g., Conv2_x), convolutional network layer 3 (e.g., Conv3_x), convolutional network layer 4 (e.g., Conv4_x), convolutional network layer 5 (e.g., Conv5_x), convolutional network layer 6 (e.g., Pool, i.e., pooling layer) and convolutional network layer 7 (e.g., FC1, i.e., fully connected layer 1). The present application uses convolutional network layer 1 to convolutional network layer 5 as feature convolution layers for performing feature convolution processing on the image; uses convolutional network layer 6 as a feature pooling layer for performing feature pooling processing on the features to be pooled output by the feature convolution layer; uses convolutional network layer 7 as a feature connection layer for performing feature connection processing on the features to be connected output by the feature pooling layer, and outputs a 32-dimensional feature vector corresponding to the image.

[0189] As shown in Table 1, there are 64 7x7 convolution kernels in Convolutional Network Layer 1, and the stride is 2; Convolutional Network Layer 2, Convolutional Network Layer 3, Convolutional Network Layer 4 and Convolutional Network Layer 5 are all residual blocks. For example, Convolutional Network Layer 2 includes a maximum pooling layer (3x3 pooling layer with a stride of 2) and 3 residual blocks, and each residual block includes 3 layers, which may include 64 1x1 convolution kernels, 64 3x3 convolution kernels, and 256 1x1 convolution kernels. Among them, 32 in the fully connected layer 1 can be the number of categories to be learned. Please refer to the following Table 1 for details:

[0190] Table 1

[0191]

[0192] Table 2

[0193]

[0194]

[0195] Among them, the business server generates the target query features corresponding to the query image through the target image recognition model. The feature recognition processing process of the image by the historical image recognition model described above can be referred to, and it will not be repeated here.

[0196] Step S205, based on the target material features corresponding to the target material images in the target image retrieval library, obtain the associated target material images associated with the target query features from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model.

[0197] Specifically, the business server obtains an index table in the target image retrieval library; the index table includes an index key and an index value, the index key is generated by an image identifier corresponding to the target material image, and the index value is generated by a target material feature; the feature distance between the target query feature and the index value is obtained, and the index value whose feature distance is less than a feature distance threshold is determined as an associated index value; the index key corresponding to the associated index value is determined as an associated index key, and the associated target material image is obtained in the target image retrieval library according to the associated index key.

[0198] Optionally, the business server sorts the target material features corresponding to each index value in ascending order of the feature distances corresponding to each index value, and determines the target material features with preset serial numbers among the sorted target material features as associated target material features, wherein the preset serial number may represent the number of materials preset by the business server, for example, the preset serial number is 10, indicating that the first 10 target material features among the sorted target material features are determined as associated target material features; further, the target material features corresponding to the associated target material features are determined as associated target material images.

[0199] Step S206, based on the historical material features corresponding to the historical material images in the historical image retrieval library, obtain the associated historical material images associated with the target local features from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model.

[0200] For details, please refer to Figure 6In the embodiment of the present application, there are two index tables, namely, an index table 501e for target material features corresponding to target material images, and an index table 501d for historical material features corresponding to historical material images, and the index table 501d is stored in the historical image retrieval library 50d. The data generation process in the index table 501d may include: the business server uses the image identifier corresponding to the historical material image as the index key and the historical material feature as the index value, so that an index key-value pair including the index key and the index value can be generated, and further, the generated index key-value pair can be stored in the index table 501d in the historical image retrieval library 50d.

[0201] The business server obtains the index table 501d in the historical image retrieval library 50d; the index table 501d includes an index key and an index value, the index key is generated by the image identifier corresponding to the historical material image, and the index value is generated by the historical material feature; the feature distance between the target local feature and the index value is obtained, and the index value whose feature distance is less than the feature distance threshold is determined as the associated index value; the index key corresponding to the associated index value is determined as the associated index key, and the associated historical material image is obtained in the historical image retrieval library 50d according to the associated index key.

[0202] Optionally, the business server sorts the historical material features corresponding to each index value in ascending order of the feature distances corresponding to each index value, and determines the historical material features belonging to a preset serial number among the sorted historical material features as associated historical material features, wherein the preset serial number may represent the number of materials preset by the business server. For example, the preset serial number is 20, indicating that the first 20 historical material features among the sorted historical material features are determined as associated historical material features; further, the historical material features corresponding to the associated historical material features are determined as associated historical material images.

[0203] Step S207: determining a recommended material image for responding to the query image from among the associated historical material images and the associated target material images.

[0204] Specifically, the business server determines the associated historical material image and the associated target material image as candidate material images; the number of candidate material images is at least two; the feature distance between the associated material features of each candidate material image and the target query feature is obtained; at least two candidate material images are sorted according to the feature distance corresponding to each candidate material image; and a recommended material image for responding to the query image is determined from the at least two sorted candidate material images.

[0205] Optionally, the service server determines the candidate material image whose feature distance is equal to the feature distance threshold as the recommended material image. It can be understood that in the present application, the feature distance thresholds in different steps can be the same or can be the same.

[0206] To sum up, the embodiments of the present application can not only generate target query features that record more image information, thereby improving the retrieval accuracy for the query image; it can also support the retrieval of related historical material images through target local features, so when there is a historical image retrieval library, it can avoid updating the historical material features in the historical material image library, thereby reducing the consumption of computing resources and reducing the update time cost.

[0207] See also Figure 7 , Figure 7 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Figure 7 As shown, the method may include at least the following steps.

[0208] Step S301: acquiring at least two target material images, and inputting the at least two target material images into a target image recognition model respectively.

[0209] Step S302: Perform feature recognition processing on at least two target material images respectively through a target image recognition model to generate target material features corresponding to the at least two target material images respectively.

[0210] Specifically, in combination with step S301 to step S302 , the target material image includes at least two target material images, and the target material feature includes target material features corresponding to the at least two target material images.

[0211] The specific process of the business server generating at least two target material features corresponding to each of the target material images can be found in the above Figure 3 The description of the target query features corresponding to the generated query image in the corresponding embodiment is that the difference between the two is that the images recognized by the target image recognition model are different, and the recognition processing process is the same, so it is not repeated here; in addition, see the above Figure 5 In the corresponding embodiments, the description of the target query features corresponding to the generated query image is different only in that the images recognized by the target image recognition model are different, and the recognition processing process is the same.

[0212] Step S303, clustering the at least two target material features to generate A target feature clusters and target cluster center features corresponding to the A target feature clusters; A is a positive integer greater than 1, and A is less than or equal to the total number of the at least two target material features.

[0213] For details, please refer to Figure 8 , Figure 8 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 8 As shown, in the embodiment of the present application, at least two target material images 70e may include image 701e, image 702e, ..., image 703e, and at least two target material features may include target material feature 701f corresponding to image 701e, target material feature 702f corresponding to image 702e, ..., target material feature 703f corresponding to image 703e, wherein at least two target material images 70e and at least two target material features may store target image retrieval library 70f. Optionally, at least two target material images 70e may be stored in a cloud server or other device, respectively. In this case, target image retrieval library 70f stores data associated with storage addresses corresponding to at least two target material images 70e (e.g., image identifiers corresponding to target material features).

[0214] Further, the service server performs clustering processing on at least two target material features to generate at least one target feature clustering cluster and at least one target clustering center feature corresponding to each target feature clustering cluster, such as Figure 8 As shown, at least one target feature clustering cluster may include a first target feature clustering cluster composed of target material features 701f, ..., target material features 703f, ..., a second target feature clustering cluster composed of at least target material features 702f, and at least one target clustering center feature may include a target clustering center feature 701g corresponding to the first target feature clustering cluster, ..., a target clustering center feature 701h corresponding to the second target feature clustering cluster. It can be understood that the feature dimensions corresponding to the target clustering center feature 701g and the target clustering center feature 701h, respectively, are equal to the feature dimensions corresponding to the target material features.

[0215] Please see again Figure 8, the business server can obtain at least two historical material images 70a, the at least two historical material images 70a can include image 701a, image 702a, ..., image 703a, and the historical material features corresponding to the at least two historical material images 70a can include historical material features 701b corresponding to image 701a, historical material features 702b corresponding to image 702a, ..., historical material features 703b corresponding to image 703a, wherein the at least two historical material images 70a and the at least two historical material features can be stored in the historical image retrieval library 70b. Optionally, the at least two historical material images 70a can be stored in the cloud server or other devices respectively, in which case the historical image retrieval library 70b stores the data associated with the storage address corresponding to the at least two historical material images 70a (for example, the image identifier corresponding to the historical material feature).

[0216] Further, the business server performs clustering processing on at least two historical material features to generate at least one historical feature cluster and at least one historical cluster center feature corresponding to each of the historical feature clusters, such as Figure 8 As shown, at least one historical feature clustering cluster may include a first historical feature clustering cluster composed of historical material features 701b, ..., historical material features 702b, ..., and a second historical feature clustering cluster composed of at least historical material features 703b, and at least one historical clustering center feature may include a historical clustering center feature 701c corresponding to the first historical feature clustering cluster, ..., and a historical clustering center feature 701d corresponding to the second historical feature clustering cluster. It can be understood that the feature dimensions corresponding to the historical clustering center feature 701c and the historical clustering center feature 701d, respectively, are equal to the feature dimensions corresponding to the historical material features.

[0217] It should be noted that the embodiments of the present application do not limit the clustering method, and K-means clustering algorithm, hierarchical agglomerative clustering algorithm and maximum-minimum distance clustering algorithm may be used. The appropriate clustering algorithm should be determined according to the actual application scenario.

[0218] Step S304: associate and store the A target feature clusters and the A target cluster center features in a target image retrieval library.

[0219] Specifically, A target feature clusters include target feature clusters C d , d is a positive integer and d is less than or equal to A. Cluster C from the target feature d Get the target material feature E f , f is a positive integer, and f is less than or equal to the target feature cluster C dThe total number of target material features in; Get the target material feature E f Corresponding target material image B f , get the target material image B f Corresponding image identification; the target material image B f The corresponding image identifier is used as the index key, and the target material feature E f As the index value, generate the cluster C with the target feature according to the index key and index value. d The associated index table; each target cluster center feature and the index table corresponding to each target cluster center feature are associated and stored in the target image retrieval library.

[0220] The business server creates an index for each target material feature in each target feature cluster. It can be understood that the specific process of creating an index for each target material feature in each target feature cluster in this step is the same as the above process. Figure 5 The process of establishing the index table in step S203 in the corresponding embodiment is consistent, so it will not be described in detail here. Please refer to the description of step S203 above.

[0221] Please combine Figure 8 And in step S303, the business server can generate an index table 702g associated with the first target feature clustering cluster, and the index table 702g can include an index key-value pair with the target material feature 701f as the index value and the image identifier corresponding to the image 701e as the index key, ..., an index key-value pair with the target material feature 703f as the index value and the image identifier corresponding to the image 703e as the index key; the business server can generate an index table 702h associated with the second target feature clustering cluster, and the index table 702h can at least include an index key-value pair with the target material feature 702f as the index value and the image identifier corresponding to the image 702e as the index key. Further, the business server associates the target cluster center feature 701g and the index table 702g and stores them in the target image retrieval library 70f, ..., and associates the target cluster center feature 701h and the index table 702h and stores them in the target image retrieval library 70f. It can be understood that Figure 8 For the sake of simplicity and aesthetics, index keys are not marked.

[0222] Please see again Figure 8 The business server creates an index for each historical material feature in each historical feature cluster. It can be understood that the specific process of creating an index for each historical material feature in each historical feature cluster in this step is the same as the above. Figure 5 The process of establishing the index table in step S203 in the corresponding embodiment is consistent, so it will not be described in detail here. Please refer to the description of step S203 above.

[0223] Please combine Figure 8 And step S303, the business server can generate an index table 702c associated with the first historical feature cluster, and the index table 702c can include an index key-value pair with the historical material feature 701b as the index value and the image identifier corresponding to the image 701a as the index key, ..., an index key-value pair with the historical material feature 703b as the index value and the image identifier corresponding to the image 703a as the index key; the business server can generate an index table 702d associated with the second historical feature cluster, and the index table 702d can at least include an index key-value pair with the historical material feature 702b as the index value and the image identifier corresponding to the image 702a as the index key. Further, the business server associates the historical cluster center feature 701c and the index table 702c and stores them in the historical image retrieval library 70b, ..., and associates the historical cluster center feature 701d and the index table 702d and stores them in the historical image retrieval library 70b.

[0224] Step S305, obtain a query image, and generate a target query feature corresponding to the query image through a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0225] Expanding the dimension of the feature vector of the image feature extraction model is a model optimization method. Since the expanded feature dimension is higher and can record more image information, the expanded feature vector can improve the retrieval accuracy. However, in the case of large-scale retrieval inventory, the problem faced by feature expansion is that the low-dimensional material features stored in the historical material inventory need to be updated to high-dimensional material features before the retrieval of high-dimensional query features can be supported. This update method consumes a lot of processor computing resources and update time, which is not business-friendly. In addition, the inventory of materials is increasing day by day, and the old material inventory will gradually be eliminated (for example, for a month's inventory, when this month's inventory comes, the previous inventory will be invalid and no longer support retrieval). Fully updating the material features in the old inventory often leads to the problem that the material features are eliminated shortly after the update, so the update benefits are not great. The embodiment of the present application proposes a solution: while maintaining the low-dimensional material features of the historical material inventory (i.e., the historical material features), the high-dimensional material features (i.e., the target material features) corresponding to the new materials (which can be equivalent to the target material images in the present application) in the new inventory (i.e., the target image retrieval library) are obtained, and at the same time, the retrieval of low-dimensional material features by the target local features in the high-dimensional query features corresponding to the query image is supported.

[0226] Compared with the historical image recognition model, the target image recognition model in the embodiment of the present application can generate high-dimensional features (including high-dimensional material features and high-dimensional query features), and some dimensional features (i.e., target local features) of the high-dimensional query features (equal to the above-mentioned target query features) remain relatively consistent with the historical query features. Therefore, the present application proposes a retrieval method for the target query features that provides new retrieval effects while supporting historical material features when the feature dimension is upgraded.

[0227] For the specific implementation process of step S305, please refer to the above Figure 3 The corresponding step S101 in the embodiment is not described in detail here.

[0228] Step S306, based on the target material features corresponding to the target material images in the target image retrieval library, obtain the associated target material images associated with the target query features from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model.

[0229] Specifically, A target cluster center features are obtained in the target image retrieval library; the A target cluster center features include target cluster center feature I j , j is a positive integer and j is less than or equal to A; obtain the target query feature and the target cluster center feature I j The characteristic distance K between j ; Sort the feature distances corresponding to A target cluster center features respectively, and determine the target feature distance from the sorted feature distances; determine the target cluster center feature corresponding to the target feature distance as the associated target cluster center feature, and obtain the associated index table stored in association with the associated target cluster center feature; according to the associated index table, obtain the associated target material image associated with the target query feature.

[0230] Step S307, based on the historical material features corresponding to the historical material images in the historical image retrieval library, obtain the associated historical material images associated with the target local features from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model.

[0231] In conjunction with step S306 and step S307, please refer to Figure 8 , the business server obtains the query image 70i, and generates a target query feature 70j corresponding to the query image 70i through the target image recognition model. The target query feature 70j includes a target local feature that has a similar association relationship with the historical query feature. Further, the business server obtains at least one target cluster center feature in the target image retrieval library 70f, such as Figure 8The target cluster center features 701g, ..., target cluster center features 701h are shown in the example, and then the feature distances between the target cluster center features 701g and the target query features 70j, ..., the feature distances between the target cluster center features 701h and the target query features 70j are obtained; further, the multiple feature distances are sorted, and the target feature distance is determined from the sorted feature distances; the business server determines the target cluster center feature corresponding to the target feature distance as the associated target cluster center feature, such as Figure 8 As shown, the target cluster center feature 701g is determined as the associated target cluster center feature, and further, the associated index table associated with the associated target cluster center feature, that is, index table 702g, is obtained. Subsequently, the business server performs feature distance calculation on the target material feature and the target query feature contained in the index table 702, that is, calculates the similarity between the target material feature and the target query feature. It can be understood that this process is similar to the above process. Figure 3 The process of step S102 in the corresponding embodiment is consistent, so it will not be described here in detail.

[0232] Further, the service server obtains at least one historical cluster center feature in the historical image retrieval library 70b, such as Figure 8 The historical cluster center features 701c, ..., historical cluster center features 701d are shown in the example, and then the feature distances between the historical cluster center features 701c and the target local features are obtained, ..., the feature distances between the historical cluster center features 701d and the target local features are obtained; further, the multiple feature distances are sorted, and the target feature distance is determined from the sorted feature distances; the business server determines the historical cluster center feature corresponding to the target feature distance as the associated historical cluster center feature, such as Figure 8 As shown, the historical cluster center feature 701c is determined as the associated historical cluster center feature, and further, the associated index table associated with the associated historical cluster center feature, that is, index table 702c, is obtained. Subsequently, the business server performs feature distance calculation on the historical material features and the target local features contained in the index table 702, that is, calculates the similarity between the historical material features and the target local features. It can be understood that this process is similar to the above Figure 3 The process of step S102 in the corresponding embodiment is consistent, so it will not be described here in detail.

[0233] Step S308: Determine a recommended material image for responding to the query image from among the associated historical material images and the associated target material images.

[0234] The specific implementation process of step S308 can be found in the above Figure 3 The description of step S104 in the corresponding embodiment is not repeated here. Figure 8The example determines images 703a, ..., image 702e as recommended material images for responding to the query image.

[0235] The embodiment of the present application maintains the original indexing function for the historical image retrieval library and forms a dual indexing system with the target image retrieval library. When the target image retrieval library accumulates sufficient inventory (which can be accumulation in time or accumulation in the number of materials, which is not limited here), the historical image retrieval library will be eliminated. After elimination, only the target image retrieval library will remain in the indexing system.

[0236] To sum up, the embodiments of the present application can not only generate target query features that record more image information, thereby improving the retrieval accuracy for the query image; it can also support the retrieval of related historical material images through target local features, so when there is a historical image retrieval library, it can avoid updating the historical material features in the historical material image library, thereby reducing the consumption of computing resources and reducing the update time cost.

[0237] See also Fig. 9 , Fig. 9 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Fig. 9 As shown, the method may include at least the following steps.

[0238] Step S401, obtaining a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image.

[0239] Specifically, at least two positive sample pairs are obtained, and at least two positive sample pairs are equally divided to obtain T positive sample pair sets; T is a positive integer, and T is less than or equal to the total number of at least two positive sample pairs; the T positive sample pair sets include the positive sample pair set U v , v is a positive integer and v is less than or equal to T; the positive sample pair set U v The target positive sample pair includes a target reference sample image and a target positive sample image corresponding to the target reference sample image; the sample image set includes a positive sample pair set U v The sample image set O in p , p is a positive integer, and p is less than or equal to the set of positive sample pairs U vThe total number of sample image sets in the U; obtaining the image distance between the target reference sample image and the remaining sample images; the remaining sample images refer to the remaining reference sample images in the remaining positive sample pairs, or the remaining positive sample images in the remaining positive sample pairs; the remaining positive sample pairs include the positive sample pair set U v positive sample pairs other than the target positive sample pairs; sorting the acquired image distances, and determining the target residual sample images from the residual sample images according to the sorted image distances; determining the target residual sample images as the target negative sample images corresponding to the target reference sample images, and determining the target negative sample images and the target positive sample pairs as the sample image set O. p .

[0240] The business server obtains the original data. The original data in the embodiment of the present application is at least two positive sample pairs. The positive sample pairs refer to two images marked as the same or similar, which are used as the training data set. Further, the business server divides the at least two positive sample pairs obtained into T positive sample pair sets. For the convenience of description and understanding, it is assumed that there are 200,000 positive sample pairs in total, and every 200 positive sample pairs are divided into a positive sample pair set, so there are 1,000 positive sample sets in total. For each positive sample pair set, any one of the samples is obtained, such as sample x, and the remaining positive sample pairs are obtained from the positive sample pair set, that is, the remaining 199 positive sample pairs that do not include sample x are obtained, and a sample image (which can be a reference sample image or a positive sample image, which is not limited here) is obtained from the remaining positive sample pairs as the remaining sample image; further, the business server calculates the image distance between sample x and the remaining sample image. The embodiment of the present application does not limit the image distance algorithm, and can be set according to the actual application scenario.

[0241] After obtaining 199 image distances, the business server processes the 199 image distances in ascending order, and determines the remaining sample images corresponding to the preset number of image distances in front as the target remaining sample images. For example, if the preset number is 10, the image distances are sorted from small to large, and the remaining sample images corresponding to the first 10 image distances are determined as the target remaining sample images; further, the business server determines the target remaining sample image as the target negative sample image of the sample x, and then the target negative sample image and the target positive sample pair including the sample x can be determined as a sample image set, which is actually a triple, including a reference sample image (Anchor), a positive sample image (Positive) and a negative sample image (Negative). It can be understood that after a positive sample pair set determines the negative sample image, it can be used as a training set, that is, 1 batch. As shown in the above numerical example, 1 batch includes 10*200 triplets.

[0242] Step S402, generating a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image through the trained historical image recognition model.

[0243] Specifically, the historical image recognition model includes a historical feature convolution layer, a historical feature pooling layer, and a historical feature connection layer. Fig.10 , Fig.10 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Fig.10 As shown, the historical image recognition model 90a includes a historical feature layer 901a and a historical feature connection layer 902a, wherein the historical feature layer 901a may include a historical feature convolution layer and a historical feature pooling layer. Fig.10 The samples in the image include a reference sample image, a positive sample image, and a negative sample image. For the specific process of the business server generating the historical reference prediction feature 901b corresponding to the reference sample image, the first historical prediction feature 902b corresponding to the positive sample image, and the second historical prediction feature 903b corresponding to the negative sample image through the historical image recognition model 90a, please refer to the above Figure 3 The description of step S101 in the corresponding embodiment is not repeated here because the model structures corresponding to the historical image recognition model 90a and the target image recognition model are consistent, and the only difference is that the number of feature connection channels in the historical feature connection layer 902a is less than the number of feature connection channels in the target feature connection layer in the target image recognition model.

[0244] Step S403, generating target baseline prediction features corresponding to the baseline sample image, first target prediction features corresponding to the positive sample image, and second target prediction features corresponding to the negative sample image through the image recognition initial model; the model parameters in the image recognition initial model include the model parameters in the historical image recognition model.

[0245] Specifically, the reference sample image, the positive sample image and the negative sample image are determined as the sample images to be processed; the sample images to be processed are input into the initial image recognition model; the initial image recognition model includes an initial feature convolution layer, an initial feature pooling layer and an initial target feature connection layer; wherein the parameters in the initial feature convolution layer are equal to the parameters in the historical feature convolution layer; the parameters in the initial feature pooling layer are equal to the parameters in the historical feature pooling layer; the initial target feature connection layer includes a historical feature connection channel and a target feature connection channel, and the parameters in the historical feature connection channel are equal to the parameters in the historical feature convolution layer; the sample images to be processed are subjected to feature convolution processing through the initial feature convolution layer to generate the sample prediction features to be pooled corresponding to the sample images to be processed, and the sample prediction features to be pooled are input into the initial feature Pooling layer; perform feature pooling processing on the prediction features of the samples to be pooled through the initial feature pooling layer to generate the prediction features of the samples to be connected corresponding to the sample image to be processed, and input the prediction features of the samples to be connected into the initial target feature connection layer; perform feature connection processing on the prediction features of the samples to be connected through the historical feature connection channel in the initial target feature connection layer to generate the historical local prediction features; perform feature connection processing on the prediction features of the samples to be connected through the target feature connection channel in the initial target feature connection layer to generate the target local prediction features; perform feature splicing processing on the historical local prediction features and the target local prediction features to obtain the target sample prediction features corresponding to the sample image to be processed; the target sample prediction features include the target baseline prediction features, the first target prediction features and the second target prediction features.

[0246] Before training the initial image recognition model, the business server first initializes the model parameters in the initial image recognition model based on the model parameters of the historical image recognition model. For ease of understanding and description, the embodiment of the present application uses the model structure example described in Table 1 above as the model structure of the historical image recognition model, and the model structure example described in Table 2 above as the model structure of the initial image recognition model. The basic preparations before training the initial image recognition model may include the following aspects.

[0247] 1) Parameter initialization: Conv1-Conv5 in Table 2 is initialized using the parameters of Conv1-Conv5 of the historical image recognition model; please refer to Fig.10 The initial target feature connection layer of the initial image recognition model includes a historical feature connection channel 902c and a target feature connection channel 903c. The parameters in the historical feature connection channel are equal to the parameters in the historical feature convolution 902a. The target feature connection channel 903c uses random numbers as initialization, such as random numbers generated by a normal distribution.

[0248] 2) Setting learning parameters: The learning parameters are set to the parameters in Conv1-Conv5 in Table 2, and the parameters in the target feature connection channel 903c. The historical feature connection channel 902c is not updated in the embodiment of the present application; Optionally, the learning parameters are set to all the parameters in Table 2. In this case, please refer to the following for the specific parameter update process. Fig.11 The description in the corresponding embodiment will not be described in the present application embodiment for the time being.

[0249] 3) Learning rate: It is feasible, learning rate = 0.0005. The learning rate is an adjustable parameter and can be adjusted according to the actual application scenario.

[0250] 4) Learning process: The data set is iterated for a preset epoch, and each epoch processes all samples. For example, in step S401, there are 200 batches in total, and each time a batch is trained, the set learning parameters are updated. When 200 batches are trained, the business server completes a model iteration.

[0251] Please see again Fig.10 The image recognition initial model 90c includes an initial feature layer 901c and an initial target feature connection layer, wherein the initial feature layer 901c may include an initial feature convolution layer and an initial feature pooling layer. The specific process of the service server generating the target reference prediction feature 901d corresponding to the reference sample image, the first target prediction feature 902d corresponding to the positive sample image, and the second target prediction feature 903d corresponding to the negative sample image through the image recognition initial model 90c is shown above. Figure 3 The description of step S101 in the corresponding embodiment is not repeated here.

[0252] Step S404, according to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature, the model parameters in the initial image recognition model are adjusted to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; the target query feature contains local features that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0253] Specifically, obtain a first feature distance between a historical benchmark prediction feature and a first historical prediction feature, and obtain a second feature distance between a historical benchmark prediction feature and a second historical prediction feature; determine a historical model loss value corresponding to the historical image recognition model based on the first feature distance and the second feature distance; obtain a third feature distance between a target benchmark prediction feature and a first target prediction feature, and obtain a fourth feature distance between the target benchmark prediction feature and the second target prediction feature; obtain a distance threshold generated based on the historical model loss value, and determine a total loss value corresponding to the image recognition initial model based on the third feature distance, the fourth feature distance, and the distance threshold; adjust parameters in the initial feature convolution layer, parameters in the initial feature pooling layer, and parameters in the target feature connection channel based on the total loss value to generate a target image recognition model.

[0254] The present application does not limit the algorithm for obtaining the feature distance between two features, which can be any distance algorithm (or similarity algorithm). The present embodiment can use formula (1) to calculate the feature distance in the embodiment of the present application, as well as the loss value (including the historical model loss value and the total model loss value involved in the embodiment of the present application).

[0255] L = max(||x a -x p ||-||x a -x n ||+α,0) (1)

[0256] Please see again Fig.10 When the business server calculates the historical model loss value, x in formula (1) a can represent the historical benchmark prediction characteristics, x p can represent the first historical prediction feature, x n It can represent the second historical prediction feature, α is a parameter in the historical image recognition model 90a, and the parameter can be set in advance. Therefore, through formula (1), the historical model loss value can be obtained.

[0257] The business server obtains the distance threshold generated based on the historical model loss value. Assuming that the number of feature connection channels corresponding to the historical feature connection layer is 32 channels, and the number of feature connection channels corresponding to the initial target feature connection layer is 64 dimensions, at this time, the number of feature connection channels corresponding to the initial target feature connection layer is twice the number of feature connection channels corresponding to the historical feature connection layer. When calculating the model loss value for the initial image recognition model 90c, α in formula (1) can be set to A times the historical model loss value, where A∈(1,2). When the business server calculates the total loss value, x in formula (1) a can represent the target benchmark prediction feature, x pCan represent the first target prediction feature, x n It can represent the second target prediction feature, so through formula (1), the total loss value for the image recognition initial model 90c can be obtained.

[0258] Furthermore, the service server can adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel 903c according to the total loss value. Specifically, the stochastic gradient descent method can be used to perform gradient backward calculation on the total loss value loss of the previous step, and the updated value of the model parameter is obtained, and then the network is updated. Fig.10 shown.

[0259] The business server records the average loss of the model in each epoch. When the average loss of the model does not decrease for 5 consecutive rounds (epochs), the model training is stopped. Optionally, when the average loss in each epoch meets formula (1), that is, when the total loss value is equal to 0, the model training is stopped. Optionally, when the number of epochs of model training is equal to the preset number, the model training is stopped.

[0260] The embodiment of the present application provides a model that supports the target query feature to include a target local feature that has a similar association relationship with the historical query feature. The prediction feature obtained by specifying the historical feature connection channel in the initial target feature connection layer in the training supports the similarity measurement task of the prediction feature obtained by the historical feature connection layer, and at the same time retains the similarity measurement ability of the prediction feature obtained by the initial target feature connection layer, so as to ensure that the target query feature (compared to the historical query feature, it belongs to a high-dimensional feature) has a partial dimension (i.e., the target local feature) that can be used for low-dimensional material feature retrieval. When applied, the target query feature or the target local feature can be selected according to the dimension of the material feature for similarity calculation, so as to realize the retrieval of the historical image retrieval library after the dimension expansion, and while reducing the feature update cost, the subsequent new inventory (i.e., the target image retrieval library) and the new retrieval feature (i.e., the target material feature) can be upgraded to the new dimension. In summary, the present application can improve the retrieval accuracy for the query image, and can also reduce the consumption of computing resources and reduce the update time cost.

[0261] See also Fig.11 , Fig.11 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Fig.11 As shown, the method may include at least the following steps.

[0262] Step S501, obtaining a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image.

[0263] Step S502, generating a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image through the trained historical image recognition model.

[0264] Step S503, generating target baseline prediction features corresponding to the baseline sample image, first target prediction features corresponding to the positive sample image, and second target prediction features corresponding to the negative sample image through the image recognition initial model; the model parameters in the image recognition initial model include the model parameters in the historical image recognition model.

[0265] The specific implementation process of steps S501 to S503 can be found in the above Fig. 9 The description of steps S401 to S403 in the corresponding embodiment will not be repeated here.

[0266] Step S504, determining the historical model loss value corresponding to the historical image recognition model according to the historical benchmark prediction feature, the first historical prediction feature and the second historical prediction feature.

[0267] Specifically, a first feature distance between a historical benchmark prediction feature and a first historical prediction feature is obtained, and a second feature distance between a historical benchmark prediction feature and a second historical prediction feature is obtained; and a historical model loss value corresponding to the historical image recognition model is determined based on the first feature distance and the second feature distance.

[0268] Step S505, determining a first loss value corresponding to the image recognition initial model according to the historical model loss value, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature.

[0269] Specifically, the third feature distance between the target reference prediction feature and the first target prediction feature is obtained, and the fourth feature distance between the target reference prediction feature and the second target prediction feature is obtained; a distance threshold generated based on the historical model loss value is obtained, and the first loss value corresponding to the image recognition initial model (equal to the above) is determined according to the third feature distance, the fourth feature distance and the distance threshold. Fig. 9The total loss value in the corresponding embodiment).

[0270] The specific implementation process of step S504-step S505 can be found in the above Fig. 9 The description of step S404 in the corresponding embodiment will not be repeated here.

[0271] Step S506, obtaining historical local prediction features, and determining a second loss value corresponding to the image recognition initial model according to the historical local prediction features and the historical model loss value.

[0272] Specifically, the historical local prediction features include historical reference local prediction features corresponding to the reference sample image, first historical local prediction features corresponding to the positive sample image, and second historical local prediction features corresponding to the negative sample image. Determine the historical local prediction feature to be replaced from the historical benchmark local prediction feature, the first historical local prediction feature and the second historical local prediction feature; when the historical local prediction feature to be replaced is the historical benchmark local prediction feature, obtain the fifth feature distance between the historical benchmark local prediction feature and the first historical local prediction feature, obtain the sixth feature distance between the historical benchmark local prediction feature and the second historical local prediction feature, and determine the second loss value corresponding to the initial model of image recognition based on the fifth feature distance, the sixth feature distance and the historical model loss value; when the historical local prediction feature to be replaced is the first historical local prediction feature, obtain the seventh feature distance between the historical benchmark local prediction feature and the first historical prediction feature, obtain the eighth feature distance between the historical benchmark local prediction feature and the second historical local prediction feature, and determine the second loss value corresponding to the initial model of image recognition based on the seventh feature distance, the eighth feature distance and the historical model loss value; when the historical local prediction feature to be replaced is the second historical local prediction feature, obtain the ninth feature distance between the historical benchmark local prediction feature and the first historical local prediction feature, obtain the tenth feature distance between the historical benchmark local prediction feature and the second historical prediction feature, and determine the second loss value corresponding to the initial model of image recognition based on the ninth feature distance, the tenth feature distance and the historical model loss value.

[0273] It should be noted that the embodiment of the present application does not set the channel order of the historical feature connection channel and the target feature connection channel. It only needs to be a continuous channel. Optionally, the historical feature connection channel can be located before the target feature connection channel. In this case, the historical local prediction feature is the leading dimension (equal to the number of historical feature connection channels) feature in the target sample prediction feature; optionally, the historical feature connection channel can be located after the target feature connection channel. In this case, the historical local prediction feature is the trailing dimension (equal to the number of historical feature connection channels) feature in the target sample prediction feature; optionally, the historical feature connection channel can be located in the middle of the target feature connection channel. In this case, the historical local prediction feature is the middle dimension (equal to the number of historical feature connection channels) feature in the target sample prediction feature.

[0274] Please also see Fig.12 , Fig.12 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Fig.12 As shown, for the convenience of description and understanding, the embodiment of the present application exemplifies the historical feature connection channel 902e, the target feature connection channel 903e and the historical feature connection layer 902a as 32 channels, and the historical feature connection channel 902e is located before the target feature connection channel 903e. It can be understood that at this time, the historical benchmark local prediction feature 901f, the first historical local prediction feature 902f, the second historical local prediction feature 903f, the historical benchmark prediction feature 901b, the first historical prediction feature 902b and the second historical prediction feature 903b are all 32-dimensional vectors, and the historical benchmark local prediction feature 901f, the first historical local prediction feature 902f and the second historical local prediction feature 903f are equivalent to Fig.12 The historical local prediction features in the dashed box.

[0275] The second loss value in the embodiment of the present application is compatible with the historical prediction features (including the historical benchmark prediction feature 901b, the first historical prediction feature 902b and the second historical prediction feature 903b) output by the historical image recognition model 90a, and the historical local prediction features (the historical benchmark local prediction feature 901f, the first historical local prediction feature 902f and the second historical local prediction feature 903f) output by the image recognition initial model 90e. Specifically, it can be as follows: first, the historical benchmark local prediction feature 901f, the first historical local prediction feature 902f and the second historical local prediction feature 903f are obtained, and the business server randomly determines one of the above three features as the historical local prediction feature to be replaced. For example, the historical benchmark local prediction feature 901f is determined as the historical local prediction feature to be replaced. Then, the second loss value can be determined by the historical benchmark prediction feature 901b, the first historical local prediction feature 902f and the second historical local prediction feature 903f according to the formula (1) above. At this time, the business server can set α in formula (1) as the historical model loss value.

[0276] Step S507: Determine the sum of the first loss value and the second loss value as the total loss value of the image recognition initial model.

[0277] Step S508, counting the number of historical adjustments of the initial image recognition model; the number of historical adjustments is used to represent the number of times the model parameters in the initial image recognition model have been adjusted.

[0278] Step S509, adjusting the model parameters in the initial image recognition model according to the total loss value and the number of historical adjustments, to generate a target image recognition model.

[0279] Specifically, when the number of historical adjustments does not meet the heartbeat update cycle, the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel are adjusted according to the total loss value, and the target image recognition model is obtained based on the adjusted image recognition initial model; when the number of historical adjustments meets the heartbeat update cycle, the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, the parameters in the target feature connection channel, and the parameters in the historical feature connection channel are adjusted according to the total loss value, and the target image recognition model is obtained based on the adjusted image recognition initial model.

[0280] Please see again Fig.12, the embodiment of the present application updates all parameters in the image recognition initial model 90e, but the historical feature connection channel 902e is regularly updated through the heartbeat mechanism (i.e., the heartbeat cycle). A feasible solution is as follows: the business server sets the heartbeat cycle to a few tenths of the number of full batches in 1 epoch. For example, if the full batch is 200, the parameters in the historical feature connection channel 902e can be updated using the total loss value after learning every 100 batches. The present application adopts the historical prediction features of the historical image recognition model to update the network weights of the historical feature connection channel 902e in the image recognition initial model 90e. By copying the weights and fine-tuning the historical local prediction features with the help of triplet loss, the historical local prediction features can meet the constraints of triplet loss with the historical prediction features under the network update, and achieve partial dimensional compatibility; at the same time, more image representation information is learned with the help of the target local prediction features, and the retrieval effect is better.

[0281] The above description is to update the parameters in the historical feature connection channel 902e (which is regularly updated), the target feature connection channel 903e, and the initial feature layer 901e (including the initial feature convolution layer and the initial feature pooling layer) through the total loss value (including the first loss value and the second loss value). Optionally, the business server updates the parameters in the target feature connection channel 903e and the initial feature layer 901e through the first loss value, and regularly updates the parameters in the historical feature connection channel 902e through the total loss value.

[0282] The embodiment of the present application provides a model that supports the target query feature to include a target local feature that has a similar association relationship with the historical query feature. The prediction feature obtained by specifying the historical feature connection channel in the initial target feature connection layer in the training supports the similarity measurement task of the prediction feature obtained by the historical feature connection layer, and at the same time retains the similarity measurement ability of the prediction feature obtained by the initial target feature connection layer, so as to ensure that the target query feature (compared to the historical query feature, it belongs to a high-dimensional feature) has a partial dimension (i.e., the target local feature) that can be used for low-dimensional material feature retrieval. When applied, the target query feature or the target local feature can be selected according to the dimension of the material feature for similarity calculation, so as to realize the retrieval of the historical image retrieval library after the dimension expansion, and while reducing the feature update cost, the subsequent new inventory (i.e., the target image retrieval library) and the new retrieval feature (i.e., the target material feature) can be upgraded to the new dimension. In summary, the present application can improve the retrieval accuracy for the query image, and can also reduce the consumption of computing resources and reduce the update time cost.

[0283] For further information, see Fig.13 , Fig.13 : is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig.13 As shown, the data processing device 1 may include: a first acquisition module 11 , a second acquisition module 12 and a material determination module 13 .

[0284] The first acquisition module 11 is used to acquire a query image and generate a target query feature corresponding to the query image through a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model;

[0285] The second acquisition module 12 is used to acquire the associated target material image associated with the target query feature from the target image retrieval library based on the target material feature corresponding to the target material image in the target image retrieval library; the target material feature refers to the feature corresponding to the target material image generated by the target image recognition model;

[0286] The second acquisition module 12 is further used to acquire the associated historical material image associated with the target local feature from the historical image retrieval library based on the historical material feature corresponding to the historical material image in the historical image retrieval library; the historical material feature refers to the feature corresponding to the historical material image generated by the historical image recognition model;

[0287] The material determination module 13 is used to determine a recommended material image for responding to the query image from the associated historical material images and the associated target material images.

[0288] The specific functional implementation of the first acquisition module 11, the second acquisition module 12 and the material determination module 13 can be referred to in the above Figure 3 Steps S101 to S104 in the corresponding embodiment will not be described in detail here.

[0289] See also Fig.13 The first acquisition module 11 may include: a first input unit 111, a second input unit 112, a first generation unit 113 and a second generation unit 114.

[0290] The first input unit 111 is used to input the query image into the target image recognition model; the target image recognition model includes a feature convolution layer, a feature pooling layer and a target feature connection layer; the number of feature connection channels corresponding to the target feature connection layer is greater than the number of feature connection channels corresponding to the historical feature connection layer in the historical image recognition model;

[0291] The second input unit 112 is used to perform feature convolution processing on the query image through the feature convolution layer, generate query features to be pooled corresponding to the query image, and input the query features to be pooled into the feature pooling layer;

[0292] The first generating unit 113 is used to perform feature pooling processing on the query features to be pooled through the feature pooling layer, generate query features to be connected corresponding to the query image, and input the query features to be connected to the target feature connection layer;

[0293] The second generating unit 114 is configured to perform feature connection processing on the query feature to be connected through the target feature connection layer to generate the target query feature.

[0294] The specific functional implementation of the first input unit 111, the second input unit 112, the first generating unit 113 and the second generating unit 114 can refer to the above Figure 5 The step S204 in the corresponding embodiment will not be described in detail here.

[0295] See also Fig.13 The data processing device 1 may further include: a third acquisition module 14 , a first generation module 15 and a second generation module 16 .

[0296] The third acquisition module 14 is used to acquire a target material image and an image identifier corresponding to the target material image;

[0297] The first generating module 15 is used to input the target material image into the target image recognition model, perform feature recognition processing on the target material image through the target image recognition model, and generate target material features;

[0298] The second generating module 16 is used to use the image identifier corresponding to the target material image as the index key and the target material feature as the index value, generate an index table according to the index key and the index value, and store the index table in the target image retrieval library.

[0299] The specific functional implementation of the third acquisition module 14, the first generation module 15 and the second generation module 16 can be found in the above Figure 5 Steps S201 to S203 in the corresponding embodiment are not described in detail here.

[0300] See also Fig.13The second acquisition module 12 may include: a first acquisition unit 121, a first determination unit 122 and a second determination unit 123.

[0301] The first acquisition unit 121 is used to acquire an index table in the target image retrieval library; the index table includes an index key and an index value, the index key is generated by an image identifier corresponding to the target material image, and the index value is generated by a target material feature;

[0302] A first determining unit 122 is used to obtain a target query feature and a feature distance between index values, and determine an index value whose feature distance is less than a feature distance threshold as an associated index value;

[0303] The second determining unit 123 is used to determine the index key corresponding to the associated index value as the associated index key, and obtain the associated target material image in the target image retrieval library according to the associated index key.

[0304] The specific functional implementation of the first acquisition unit 121, the first determination unit 122 and the second determination unit 123 can be referred to above. Figure 5 The step S205 in the corresponding embodiment will not be described in detail here.

[0305] See also Fig.13 , the target material image includes at least two target material images, and the target material feature includes target material features corresponding to the at least two target material images respectively;

[0306] The data processing device 1 may further include: a fourth acquisition module 17 , a third generation module 18 and a fourth generation module 19 .

[0307] A fourth acquisition module 17, used to acquire at least two target material images, and input the at least two target material images into the target image recognition model respectively;

[0308] The third generating module 18 is used to perform feature recognition processing on at least two target material images respectively through the target image recognition model to generate target material features corresponding to the at least two target material images respectively;

[0309] The fourth generating module 19 is used to perform clustering processing on at least two target material features to generate A target feature clustering clusters and target cluster center features corresponding to the A target feature clustering clusters respectively; A is a positive integer greater than 1, and A is less than or equal to the total number of the at least two target material features;

[0310] The fourth generating module 19 is further used to associate and store the A target feature clusters and the A target cluster center features in a target image retrieval library.

[0311] The specific functional implementation of the fourth acquisition module 17, the third generation module 18 and the fourth generation module 19 can be found in the above Figure 7 Steps S301 to S304 in the corresponding embodiment are not described in detail here.

[0312] See also Fig.13 , A target feature clusters include target feature clusters C d , d is a positive integer and d is less than or equal to A;

[0313] The fourth generating module 19 may include: a second acquiring unit 191 , a third acquiring unit 192 , a third generating unit 193 and an associated storage unit 194 .

[0314] The second acquisition unit 191 is used to cluster the cluster C from the target feature d Get the target material feature E f , f is a positive integer, and f is less than or equal to the target feature cluster C d The total number of target material features in ;

[0315] The third acquisition unit 192 is used to acquire the target material feature E f Corresponding target material image B f , get the target material image B f The corresponding image ID;

[0316] The third generating unit 193 is used to generate the target material image B f The corresponding image identifier is used as the index key, and the target material feature E f As the index value, generate the cluster C with the target feature according to the index key and index value. d The associated index table;

[0317] The associated storage unit 194 is used to associate and store each target cluster center feature and the index table corresponding to each target cluster center feature in the target image retrieval library.

[0318] The specific functional implementation of the second acquisition unit 191, the third acquisition unit 192, the third generation unit 193 and the associated storage unit 194 can be found in the above Figure 7 The step S304 in the corresponding embodiment will not be described in detail here.

[0319] See also Fig.13 The second acquisition module 12 may include: a fourth acquisition unit 124 , a fifth acquisition unit 125 , a third determination unit 126 , a sixth acquisition unit 127 and a seventh acquisition unit 128 .

[0320] The fourth acquisition unit 124 is used to acquire A target cluster center features in the target image retrieval library; the A target cluster center features include target cluster center features I j , j is a positive integer and j is less than or equal to A;

[0321] The fifth acquisition unit 125 is used to acquire the target query feature and the target cluster center feature I j The characteristic distance K between j ;

[0322] The third determining unit 126 is used to sort the feature distances corresponding to the A target cluster center features, and determine the target feature distance from the sorted feature distances;

[0323] The sixth acquisition unit 127 is used to determine the target cluster center feature corresponding to the target feature distance as the associated target cluster center feature, and acquire an associated index table stored in association with the associated target cluster center feature;

[0324] The seventh acquisition unit 128 is used to acquire the associated target material image associated with the target query feature according to the associated index table.

[0325] The specific functional implementation of the fourth acquisition unit 124, the fifth acquisition unit 125, the third determination unit 126, the sixth acquisition unit 127 and the seventh acquisition unit 128 can be referred to above. Figure 3 The step S306 in the corresponding embodiment will not be described in detail here.

[0326] See also Fig.13 The material determination module 13 may include: a fourth determination unit 131 , an eighth acquisition unit 132 and a fifth determination unit 133 .

[0327] The fourth determining unit 131 is used to determine the associated historical material image and the associated target material image as candidate material images; the number of candidate material images is at least two;

[0328] An eighth acquisition unit 132 is used to acquire feature distances between the associated material features of each candidate material image and the target query feature;

[0329] A fifth determining unit 133, configured to sort at least two candidate material images according to the feature distance corresponding to each candidate material image;

[0330] The fifth determining unit 133 is further configured to determine a recommended material image for responding to the query image from the at least two sorted candidate material images.

[0331] The specific functional implementation of the fourth determining unit 131, the eighth obtaining unit 132 and the fifth determining unit 133 can refer to the above Figure 5 The step S207 in the corresponding embodiment will not be described in detail here.

[0332] To sum up, the embodiments of the present application can not only generate target query features that record more image information, thereby improving the retrieval accuracy for the query image; it can also support the retrieval of related historical material images through target local features, so when there is a historical image retrieval library, it can avoid updating the historical material features in the historical material image library, thereby reducing the consumption of computing resources and reducing the update time cost.

[0333] For further information, see Fig.14 , Fig.14 : is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig.14 As shown, the data processing device 2 may include: a sample acquisition module 21 , a first generation module 22 , a second generation module 23 and a third generation module 24 .

[0334] The sample acquisition module 21 is used to acquire a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image;

[0335] A first generating module 22, used to generate a historical benchmark prediction feature corresponding to a benchmark sample image, a first historical prediction feature corresponding to a positive sample image, and a second historical prediction feature corresponding to a negative sample image through a trained historical image recognition model;

[0336] The second generation module 23 is used to generate a target reference prediction feature corresponding to the reference sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image through an image recognition initial model; the model parameters in the image recognition initial model include the model parameters in the historical image recognition model;

[0337] The third generation module 24 is used to adjust the model parameters in the image recognition initial model according to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; the target query feature contains local features that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0338] The specific functional implementation of the sample acquisition module 21, the first generation module 22, the second generation module 23 and the third generation module 24 can be found in the above Fig. 9 Steps S401 to S404 in the corresponding embodiment are not described in detail here.

[0339] See also Fig.14 The sample acquisition module 21 may include: a first acquisition unit 211 , a second acquisition unit 212 , a first determination unit 213 and a second determination unit 214 .

[0340] The first acquisition unit 211 is used to acquire at least two positive sample pairs, and perform equal division processing on the at least two positive sample pairs to obtain T positive sample pair sets; T is a positive integer, and T is less than or equal to the total number of at least two positive sample pairs; the T positive sample pair sets include the positive sample pair set U v , v is a positive integer and v is less than or equal to T; the positive sample pair set U v The target positive sample pair includes a target reference sample image and a target positive sample image corresponding to the target reference sample image; the sample image set includes a positive sample pair set U v The sample image set O in p , p is a positive integer, and p is less than or equal to the set of positive sample pairs U v The total number of sample image sets in ;

[0341] The second acquisition unit 212 is used to acquire the image distance between the target reference sample image and the remaining sample images; the remaining sample images refer to the remaining reference sample images in the remaining positive sample pairs, or the remaining positive sample images in the remaining positive sample pairs; the remaining positive sample pairs include the positive sample pair set U v Positive sample pairs other than the target positive sample pairs;

[0342] A first determining unit 213 is used to sort the acquired image distances and determine a target remaining sample image from the remaining sample images according to the sorted image distances;

[0343] The second determining unit 214 is used to determine the target residual sample image as the target negative sample image corresponding to the target reference sample image, and determine the target negative sample image and the target positive sample pair as the sample image set. p .

[0344] The specific functional implementation of the first acquisition unit 211, the second acquisition unit 212, the first determination unit 213 and the second determination unit 214 can be referred to above. Fig. 9 The step S401 in the corresponding embodiment will not be described in detail here.

[0345] See also Fig.14 ,The historical image recognition model includes a historical feature convolution layer, a historical feature pooling layer, and a historical feature connection layer;

[0346] The second generating module 23 may include: a third determining unit 231 , a first input unit 232 , a second input unit 233 , a third input unit 234 , a first processing unit 235 , a second processing unit 236 , and a third processing unit 237 .

[0347] A third determining unit 231 is used to determine the reference sample image, the positive sample image and the negative sample image as sample images to be processed;

[0348] The first input unit 232 is used to input the sample image to be processed into the image recognition initial model; the image recognition initial model includes an initial feature convolution layer, an initial feature pooling layer and an initial target feature connection layer; wherein the parameters in the initial feature convolution layer are equal to the parameters in the historical feature convolution layer; the parameters in the initial feature pooling layer are equal to the parameters in the historical feature pooling layer; the initial target feature connection layer includes a historical feature connection channel and a target feature connection channel, and the parameters in the historical feature connection channel are equal to the parameters in the historical feature convolution layer;

[0349] The second input unit 233 is used to perform feature convolution processing on the sample image to be processed through the initial feature convolution layer, generate the sample prediction features to be pooled corresponding to the sample image to be processed, and input the sample prediction features to be pooled into the initial feature pooling layer;

[0350] The third input unit 234 is used to perform feature pooling processing on the sample prediction features to be pooled through the initial feature pooling layer, generate the sample prediction features to be connected corresponding to the sample image to be processed, and input the sample prediction features to be connected to the initial target feature connection layer;

[0351] A first processing unit 235 is used to perform feature connection processing on the prediction features of the samples to be connected through the historical feature connection channel in the initial target feature connection layer to generate historical local prediction features;

[0352] The second processing unit 236 is used to perform feature connection processing on the prediction features of the samples to be connected through the target feature connection channel in the initial target feature connection layer to generate target local prediction features;

[0353] The third processing unit 237 is used to perform feature splicing processing on the historical local prediction features and the target local prediction features to obtain the target sample prediction features corresponding to the sample image to be processed; the target sample prediction features include the target reference prediction features, the first target prediction features and the second target prediction features.

[0354] The specific functional implementation of the third determining unit 231, the first input unit 232, the second input unit 233, the third input unit 234, the first processing unit 235, the second processing unit 236 and the third processing unit 237 can be referred to above. Fig. 9 The step S403 in the corresponding embodiment will not be described in detail here.

[0355] See also Fig.14 The third generating module 24 may include: a third acquiring unit 241 , a fourth determining unit 242 , a fourth acquiring unit 243 , a fifth determining unit 244 and a first generating unit 245 .

[0356] The third acquisition unit 241 is used to acquire a first feature distance between the historical benchmark prediction feature and the first historical prediction feature, and to acquire a second feature distance between the historical benchmark prediction feature and the second historical prediction feature;

[0357] A fourth determining unit 242, configured to determine a historical model loss value corresponding to the historical image recognition model according to the first feature distance and the second feature distance;

[0358] A fourth acquisition unit 243 is used to acquire a third characteristic distance between the target reference prediction feature and the first target prediction feature, and to acquire a fourth characteristic distance between the target reference prediction feature and the second target prediction feature;

[0359] A fifth determining unit 244 is used to obtain a distance threshold generated based on the historical model loss value, and determine a total loss value corresponding to the image recognition initial model according to the third feature distance, the fourth feature distance and the distance threshold;

[0360] The first generating unit 245 is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel according to the total loss value to generate a target image recognition model.

[0361] The specific functional implementation of the third acquisition unit 241, the fourth determination unit 242, the fourth acquisition unit 243, the fifth determination unit 244 and the first generation unit 245 can refer to the above Fig. 9 The step S404 in the corresponding embodiment will not be described in detail here.

[0362] See also Fig.14 The third generating module 24 may include: a sixth determining unit 246 , a seventh determining unit 247 , a statistical history unit 248 and a second generating unit 249 .

[0363] A sixth determining unit 246, configured to determine a historical model loss value corresponding to the historical image recognition model according to the historical benchmark prediction feature, the first historical prediction feature, and the second historical prediction feature;

[0364] The sixth determining unit 246 is further used to determine a first loss value corresponding to the image recognition initial model according to the historical model loss value, the target reference prediction feature, the first target prediction feature, and the second target prediction feature;

[0365] A seventh determination unit 247 is used to obtain historical local prediction features, and determine a second loss value corresponding to the image recognition initial model according to the historical local prediction features and the historical model loss value;

[0366] The seventh determining unit 247 is further used to determine the sum of the first loss value and the second loss value as the total loss value of the image recognition initial model;

[0367] A statistical history unit 248 is used to count the number of historical adjustments of the image recognition initial model; the number of historical adjustments is used to represent the number of times the model parameters in the image recognition initial model have been adjusted;

[0368] The second generating unit 249 is used to adjust the model parameters in the initial image recognition model according to the total loss value and the number of historical adjustments to generate a target image recognition model.

[0369] The specific functional implementation of the sixth determination unit 246, the seventh determination unit 247, the statistical history unit 248 and the second generation unit 249 can be referred to above. Fig.11 Steps S504 to S509 in the corresponding embodiment are not described in detail here.

[0370] See also Fig.14 , the historical local prediction features include the historical reference local prediction features corresponding to the reference sample image, the first historical local prediction features corresponding to the positive sample image, and the second historical local prediction features corresponding to the negative sample image;

[0371] The seventh determining unit 247 may include: a first determining subunit 2471 and a second determining subunit 2472 .

[0372] A first determining subunit 2471 is used to determine the historical local prediction feature to be replaced from the historical benchmark local prediction feature, the first historical local prediction feature and the second historical local prediction feature;

[0373] The second determining subunit 2472 is used to obtain, when the historical local prediction feature to be replaced is the historical benchmark local prediction feature, a fifth feature distance between the historical benchmark prediction feature and the first historical local prediction feature, obtain a sixth feature distance between the historical benchmark prediction feature and the second historical local prediction feature, and determine a second loss value corresponding to the image recognition initial model according to the fifth feature distance, the sixth feature distance and the historical model loss value;

[0374] The second determining subunit 2472 is further configured to, when the historical local prediction feature to be replaced is the first historical local prediction feature, obtain a seventh feature distance between the historical benchmark local prediction feature and the first historical prediction feature, obtain an eighth feature distance between the historical benchmark local prediction feature and the second historical local prediction feature, and determine a second loss value corresponding to the image recognition initial model according to the seventh feature distance, the eighth feature distance, and the historical model loss value;

[0375] The second determination subunit 2472 is used to obtain the ninth feature distance between the historical benchmark local prediction feature and the first historical local prediction feature when the historical local prediction feature to be replaced is the second historical local prediction feature, and to obtain the tenth feature distance between the historical benchmark local prediction feature and the second historical prediction feature, and to determine the second loss value corresponding to the image recognition initial model according to the ninth feature distance, the tenth feature distance and the historical model loss value.

[0376] The specific functional implementation of the first determining subunit 2471 and the second determining subunit 2472 can be referred to above. Fig.11 The step S506 in the corresponding embodiment will not be described in detail here.

[0377] See also Fig.14 The second generating unit 249 may include: a first adjusting subunit 2491 and a second adjusting subunit 2492 .

[0378] The first adjustment subunit 2491 is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel according to the total loss value when the historical adjustment times do not meet the heartbeat update cycle, and obtain the target image recognition model based on the adjusted image recognition initial model;

[0379] The second adjustment subunit 2492 is used to adjust the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, the parameters in the target feature connection channel, and the parameters in the historical feature connection channel according to the total loss value when the number of historical adjustments meets the heartbeat update cycle, and obtain the target image recognition model based on the adjusted image recognition initial model.

[0380] The specific functional implementation of the first adjustment subunit 2491 and the second adjustment subunit 2492 can be found in the above Fig.11 The step S509 in the corresponding embodiment will not be described in detail here.

[0381] The embodiment of the present application provides a model that supports the target query feature to include a target local feature that has a similar association relationship with the historical query feature. The prediction feature obtained by specifying the historical feature connection channel in the initial target feature connection layer in the training supports the similarity measurement task of the prediction feature obtained by the historical feature connection layer, and at the same time retains the similarity measurement ability of the prediction feature obtained by the initial target feature connection layer, so as to ensure that the target query feature (compared to the historical query feature, it belongs to a high-dimensional feature) has a partial dimension (i.e., the target local feature) that can be used for low-dimensional material feature retrieval. When applied, the target query feature or the target local feature can be selected according to the dimension of the material feature for similarity calculation, so as to realize the retrieval of the historical image retrieval library after the dimension expansion, and while reducing the feature update cost, the subsequent new inventory (i.e., the target image retrieval library) and the new retrieval feature (i.e., the target material feature) can be upgraded to the new dimension. In summary, the present application can improve the retrieval accuracy for the query image, and can also reduce the consumption of computing resources and reduce the update time cost.

[0382] For further information, see Fig.15 , Fig.15 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.15 As shown, the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. Fig.15 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0383] exist Fig.15 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0384] A query image is obtained, and a target query feature corresponding to the query image is generated by a target image recognition model; the target image recognition model is trained based on a trained historical image recognition model; the target query feature contains a target local feature that has a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to a feature corresponding to the query image generated by the historical image recognition model;

[0385] Based on the target material features corresponding to the target material images in the target image retrieval library, the associated target material images associated with the target query features are obtained from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model;

[0386] Based on the historical material features corresponding to the historical material images in the historical image retrieval library, the associated historical material images associated with the target local features are obtained from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model;

[0387] Among the association history material images and the association target material images, a recommended material image for responding to the query image is determined.

[0388] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 , Figure 5 , Figure 7 , Fig. 9 as well as Fig.11 The description of the data processing method in the corresponding embodiment can also be performed as described above. Fig.13 The description of the data processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0389] For further information, see Fig.16 , Fig.16 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.16As shown, the above-mentioned computer device 2000 may include: a processor 2001, a network interface 2004 and a memory 2005. In addition, the above-mentioned computer device 2000 may also include: a user interface 2003, and at least one communication bus 2002. Among them, the communication bus 2002 is used to realize the connection and communication between these components. Among them, the user interface 2003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 2003 may also include a standard wired interface and a wireless interface. The network interface 2004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 2005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 2005 may optionally also be at least one storage device located away from the aforementioned processor 2001. As Fig.16 As shown, the memory 2005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0390] exist Fig.16 In the computer device 2000 shown, the network interface 2004 can provide a network communication function; the user interface 2003 is mainly used to provide an input interface for the user; and the processor 2001 can be used to call the device control application stored in the memory 2005 to achieve:

[0391] Acquire a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image;

[0392] Generate a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image through a trained historical image recognition model;

[0393] Generate a target benchmark prediction feature corresponding to the benchmark sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image through an initial image recognition model; the model parameters in the initial image recognition model include the model parameters in the historical image recognition model;

[0394] According to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature, the model parameters in the initial image recognition model are adjusted to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; there are local features in the target query feature that have a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model.

[0395] It should be understood that the computer device 2000 described in the embodiment of the present application can execute the above Figure 3 , Figure 5 , Figure 7 , Fig. 9 as well as Fig.11 The description of the data processing method in the corresponding embodiment can also be performed as described above. Fig.14 The description of the data processing device 2 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0396] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, Figure 3 , Figure 5 , Figure 7 , Fig. 9 as well as Fig.11 For details on the data processing methods provided in each step, please refer to the above Figure 3 , Figure 5 , Figure 7 , Fig. 9 as well as Fig.11 The implementation methods provided by each step will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail here either.

[0397] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0398] The embodiment of the present application also provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device can execute the above Figure 3 , Figure 5 , Figure 7 , Fig. 9 as well as Fig.11 The description of the data processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here.

[0399] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0400] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0401] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0402] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Acquire a query image, and generate a target query feature corresponding to the query image through a target image recognition model; The target image recognition model is trained based on the trained historical image recognition model; the feature connection channel contained in the target image recognition model is divided into a historical feature connection channel and a target feature connection channel; the target query feature contains a target local feature having a similar correlation relationship with the historical query feature, and the feature dimension corresponding to the target query feature is greater than the feature dimension corresponding to the historical query feature, and the historical query feature refers to the feature corresponding to the query image generated by the historical image recognition model; Based on the target material features corresponding to the target material images in the target image retrieval library, an associated target material image associated with the target query features is obtained from the target image retrieval library; the target material features refer to the features corresponding to the target material images generated by the target image recognition model; the target features include the target query features and the target material features; Based on the historical material features corresponding to the historical material images in the historical image retrieval library, the associated historical material images associated with the target local features are obtained from the historical image retrieval library; the historical material features refer to the features corresponding to the historical material images generated by the historical image recognition model; the historical features include the historical query features and the historical material features; Among the association history material images and the association target material images, a recommended material image for responding to the query image is determined.

2. The method according to claim 1, characterized in that: The generating the target query feature corresponding to the query image by the target image recognition model includes: The query image is input into the target image recognition model; the target image recognition model includes a feature convolution layer, a feature pooling layer and a target feature connection layer; the number of feature connection channels corresponding to the target feature connection layer is greater than the number of feature connection channels corresponding to the historical feature connection layer in the historical image recognition model; Performing feature convolution processing on the query image through the feature convolution layer to generate query features to be pooled corresponding to the query image, and inputting the query features to be pooled into the feature pooling layer; Performing feature pooling processing on the query features to be pooled through the feature pooling layer to generate query features to be connected corresponding to the query image, and inputting the query features to be connected into the target feature connection layer; The target feature connection layer performs feature connection processing on the query feature to be connected to generate the target query feature.

3. The method according to claim 1, characterized in that The method further comprises: Acquire the target material image, and acquire the image identifier corresponding to the target material image; Inputting the target material image into the target image recognition model, performing feature recognition processing on the target material image through the target image recognition model, and generating the target material features; The image identifier corresponding to the target material image is used as an index key, the target material feature is used as an index value, an index table is generated according to the index key and the index value, and the index table is stored in the target image retrieval library.

4. The method according to claim 1, characterized in that: The step of acquiring, based on the target material feature corresponding to the target material image in the target image retrieval library, the associated target material image associated with the target query feature from the target image retrieval library comprises: Acquire an index table in the target image retrieval library; the index table includes an index key and an index value, the index key is generated by an image identifier corresponding to the target material image, and the index value is generated by a feature of the target material; Acquire the target query feature and the feature distance between the index values, and determine the index value whose feature distance is less than the feature distance threshold as the associated index value; An index key corresponding to the associated index value is determined as an associated index key, and the associated target material image is obtained in the target image retrieval library according to the associated index key.

5. The method according to claim 1, characterized in that The target material image includes at least two target material images, and the target material feature includes target material features corresponding to the at least two target material images respectively; The method further comprises: Acquire the at least two target material images, and input the at least two target material images into the target image recognition model respectively; By using the target image recognition model, feature recognition processing is performed on the at least two target material images respectively to generate target material features corresponding to the at least two target material images respectively; Performing clustering processing on at least two target material features to generate A target feature clustering clusters and target cluster center features corresponding to the A target feature clustering clusters respectively; A is a positive integer greater than 1, and A is less than or equal to the total number of the at least two target material features; The A target feature clusters and the A target cluster center features are associated and stored in the target image retrieval library.

6. The method according to claim 5, characterized in that The A target feature clusters include target feature cluster C d , d is a positive integer and d is less than or equal to A; The step of associating and storing the A target feature clusters and the A target cluster center features in the target image retrieval library comprises: Cluster C from the target features d Get the target material feature E f , f is a positive integer, and f is less than or equal to the target feature cluster C d The total number of target material features in ; Get the target material feature E f Corresponding target material image B f , obtain the target material image B f The corresponding image ID; The target material image B f The corresponding image identifier is used as the index key, and the target material feature E f As the index value, generate a cluster C corresponding to the target feature according to the index key and the index value. d The associated index table; Each target cluster center feature and the index table corresponding to each target cluster center feature are associated and stored in the target image retrieval library.

7. The method according to claim 6, characterized in that The step of acquiring, based on the target material feature corresponding to the target material image in the target image retrieval library, the associated target material image associated with the target query feature from the target image retrieval library comprises: The A target cluster center features are obtained in the target image retrieval library; the A target cluster center features include target cluster center feature I j , j is a positive integer and j is less than or equal to A; Obtain the target query feature and the target cluster center feature I j The characteristic distance K between j ; Sorting the feature distances corresponding to the A target cluster center features, and determining the target feature distance from the sorted feature distances; Determine the target cluster center feature corresponding to the target feature distance as the associated target cluster center feature, and obtain an associated index table stored in association with the associated target cluster center feature; According to the associated index table, the associated target material image associated with the target query feature is obtained.

8. The method according to claim 1, characterized in that The step of determining a recommended material image for responding to the query image from among the associated historical material images and the associated target material images includes: determining the associated historical material image and the associated target material image as candidate material images; the number of the candidate material images is at least two; Obtaining feature distances between the associated material features of each candidate material image and the target query feature; sorting the at least two candidate material images according to the feature distances respectively corresponding to each of the candidate material images; The recommended material image for responding to the query image is determined from the at least two sorted candidate material images.

9. A data processing method, characterized in that: include: Acquire a sample image set; the sample image set includes a reference sample image, a positive sample image corresponding to the reference sample image, and a negative sample image corresponding to the reference sample image; Generate, by means of a trained historical image recognition model, a historical benchmark prediction feature corresponding to the benchmark sample image, a first historical prediction feature corresponding to the positive sample image, and a second historical prediction feature corresponding to the negative sample image; Generate a target benchmark prediction feature corresponding to the benchmark sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image through an image recognition initial model; the model parameters in the image recognition initial model include the model parameters in the historical image recognition model; According to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature and the second target prediction feature, the model parameters in the image recognition initial model are adjusted to generate a target image recognition model; the target image recognition model is used to generate a target query feature corresponding to the query image; The feature connection channels contained in the target image recognition model are divided into historical feature connection channels and target feature connection channels, the historical features include historical query features, and the target features include the target query features; the target query features contain local features that have similar correlation relationships with the historical query features, and the feature dimension corresponding to the target query features is greater than the feature dimension corresponding to the historical query features, and the historical query features refer to the features corresponding to the query image generated by the historical image recognition model.

10. The method according to claim 9, characterized in that The step of obtaining a sample image set comprises: Obtain at least two positive sample pairs, and divide the at least two positive sample pairs equally to obtain T positive sample pair sets; T is a positive integer, and T is less than or equal to the total number of the at least two positive sample pairs; the T positive sample pair sets include the positive sample pair set U v , v is a positive integer and v is less than or equal to T; the positive sample pair set U v The target positive sample pair includes a target reference sample image and a target positive sample image corresponding to the target reference sample image; the sample image set includes the positive sample pair set U v The sample image set O in p , p is a positive integer, and p is less than or equal to the positive sample pair set U v The total number of sample image sets in ; Obtaining an image distance between the target reference sample image and the remaining sample image; the remaining sample image refers to the remaining reference sample image in the remaining positive sample pair, or the remaining positive sample image in the remaining positive sample pair; the remaining positive sample pair includes the positive sample pair set U v positive sample pairs other than the target positive sample pairs; Sorting the acquired image distances, and determining a target remaining sample image from the remaining sample images according to the sorted image distances; The target residual sample image is determined as the target negative sample image corresponding to the target reference sample image, and the target negative sample image and the target positive sample pair are determined as the sample image set. p .

11. The method according to claim 9, characterized in that The historical image recognition model includes a historical feature convolution layer, a historical feature pooling layer and the historical feature connection layer; The generating, by using the image recognition initial model, a target reference prediction feature corresponding to the reference sample image, a first target prediction feature corresponding to the positive sample image, and a second target prediction feature corresponding to the negative sample image, comprises: Determine the reference sample image, the positive sample image, and the negative sample image as sample images to be processed; Input the sample image to be processed into the image recognition initial model; the image recognition initial model includes an initial feature convolution layer, an initial feature pooling layer and an initial target feature connection layer; wherein the parameters in the initial feature convolution layer are equal to the parameters in the historical feature convolution layer; the parameters in the initial feature pooling layer are equal to the parameters in the historical feature pooling layer; the initial target feature connection layer includes a historical feature connection channel and a target feature connection channel, and the parameters in the historical feature connection channel are equal to the parameters in the historical feature convolution layer; Performing feature convolution processing on the sample image to be processed through the initial feature convolution layer to generate prediction features of samples to be pooled corresponding to the sample image to be processed, and inputting the prediction features of samples to be pooled into the initial feature pooling layer; Performing feature pooling processing on the sample prediction features to be pooled through the initial feature pooling layer to generate sample prediction features to be connected corresponding to the sample image to be processed, and inputting the sample prediction features to be connected into the initial target feature connection layer; Performing feature connection processing on the prediction features of the sample to be connected through the historical feature connection channel in the initial target feature connection layer to generate historical local prediction features; Performing feature connection processing on the prediction features of the samples to be connected through the target feature connection channel in the initial target feature connection layer to generate target local prediction features; The historical local prediction features and the target local prediction features are subjected to feature splicing processing to obtain the target sample prediction features corresponding to the sample image to be processed; the target sample prediction features include the target baseline prediction features, the first target prediction features and the second target prediction features.

12. The method according to claim 11, characterized in that The step of adjusting the model parameters in the initial image recognition model according to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature, and the second target prediction feature to generate a target image recognition model includes: Obtaining a first characteristic distance between the historical benchmark prediction feature and the first historical prediction feature, and obtaining a second characteristic distance between the historical benchmark prediction feature and the second historical prediction feature; Determining a historical model loss value corresponding to the historical image recognition model according to the first feature distance and the second feature distance; Obtaining a third characteristic distance between the target reference prediction feature and the first target prediction feature, and obtaining a fourth characteristic distance between the target reference prediction feature and the second target prediction feature; Obtaining a distance threshold generated based on the historical model loss value, and determining a total loss value corresponding to the image recognition initial model according to the third feature distance, the fourth feature distance, and the distance threshold; According to the total loss value, the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel are adjusted to generate the target image recognition model.

13. The method according to claim 11, characterized in that The step of adjusting the model parameters in the initial image recognition model according to the historical benchmark prediction feature, the first historical prediction feature, the second historical prediction feature, the target benchmark prediction feature, the first target prediction feature, and the second target prediction feature to generate a target image recognition model includes: Determining a historical model loss value corresponding to the historical image recognition model according to the historical benchmark prediction feature, the first historical prediction feature, and the second historical prediction feature; Determining a first loss value corresponding to the image recognition initial model according to the historical model loss value, the target benchmark prediction feature, the first target prediction feature, and the second target prediction feature; Acquire the historical local prediction feature, and determine a second loss value corresponding to the image recognition initial model according to the historical local prediction feature and the historical model loss value; Determine the sum of the first loss value and the second loss value as the total loss value of the image recognition initial model; Counting the number of historical adjustments of the initial image recognition model; the number of historical adjustments is used to represent the number of times the model parameters in the initial image recognition model have been adjusted; According to the total loss value and the number of historical adjustments, the model parameters in the initial image recognition model are adjusted to generate the target image recognition model.

14. The method according to claim 13, characterized in that The historical local prediction features include the historical reference local prediction features corresponding to the reference sample image, the first historical local prediction features corresponding to the positive sample image, and the second historical local prediction features corresponding to the negative sample image; The acquiring the historical local prediction feature and determining the second loss value corresponding to the image recognition initial model according to the historical local prediction feature and the historical model loss value includes: Determine a historical local prediction feature to be replaced from the historical benchmark local prediction feature, the first historical local prediction feature, and the second historical local prediction feature; When the historical local prediction feature to be replaced is the historical benchmark local prediction feature, obtaining a fifth feature distance between the historical benchmark prediction feature and the first historical local prediction feature, obtaining a sixth feature distance between the historical benchmark prediction feature and the second historical local prediction feature, and determining the second loss value corresponding to the image recognition initial model according to the fifth feature distance, the sixth feature distance and the historical model loss value; When the historical local prediction feature to be replaced is the first historical local prediction feature, obtaining a seventh feature distance between the historical benchmark local prediction feature and the first historical prediction feature, obtaining an eighth feature distance between the historical benchmark local prediction feature and the second historical local prediction feature, and determining the second loss value corresponding to the image recognition initial model according to the seventh feature distance, the eighth feature distance and the historical model loss value; When the historical local prediction feature to be replaced is the second historical local prediction feature, the ninth feature distance between the historical benchmark local prediction feature and the first historical local prediction feature is obtained, and the tenth feature distance between the historical benchmark local prediction feature and the second historical prediction feature is obtained. According to the ninth feature distance, the tenth feature distance and the historical model loss value, the second loss value corresponding to the image recognition initial model is determined.

15. The method according to claim 13, characterized in that The adjusting the model parameters in the initial image recognition model according to the total loss value and the number of historical adjustments to generate the target image recognition model includes: When the number of historical adjustments does not satisfy the heartbeat update cycle, adjusting the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, and the parameters in the target feature connection channel according to the total loss value, and obtaining the target image recognition model based on the adjusted image recognition initial model; When the number of historical adjustments meets the heartbeat update cycle, the parameters in the initial feature convolution layer, the parameters in the initial feature pooling layer, the parameters in the target feature connection channel, and the parameters in the historical feature connection channel are adjusted according to the total loss value, and the target image recognition model is obtained based on the adjusted image recognition initial model.

16. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 15.

18. A computer program product, characterized in that The computer program product comprises computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so as to cause a computer device having the processor to perform the method according to any one of claims 1 to 15.

Images