Content search method and device based on large model technology and electronic equipment

Abstract

The invention provides a content search method and device based on a large model technology and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, natural language processing, computer vision, large models and the like. According to the specific implementation scheme, the method comprises the steps of obtaining to-be-searched content and a search result image related to the to-be-searched content in at least one target application; the search result image is obtained by combining the to-be-searched content and a barrier-free service search corresponding to the target application; performing content identification processing on the search result image in combination with the large model to obtain identification content; performing fusion processing on the search result image and the recognition content in combination with the large model to obtain a target search result corresponding to the to-be-searched content; wherein the target search result is obtained by fusing the search result image and the corresponding identification content, the conditions of content overlapping, disordered arrangement and the like do not exist, and the required content can be effectively extracted from the target search result, so that the content search efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as deep learning, natural language processing, computer vision, and large models, and in particular to a content search method, device, and electronic device based on large model technology. Background Art

[0002] The current content search method mainly includes: for the content to be searched, the content to be searched can be input into the search box in each target application to obtain the search result image related to the content to be searched in each target application; and the search result image is used as the target search result.

[0003] In the above scheme, there are a large number of search result images, and there may be content overlap and disordered arrangement between multiple search result images, making it difficult to effectively extract required content from the multiple search result images, thereby reducing content search efficiency. Summary of the invention

[0004] The present disclosure provides a content search method, device and electronic device based on big model technology.

[0005] According to one aspect of the present disclosure, a content search method based on big model technology is provided, the method comprising: obtaining content to be searched and a search result image related to the content to be searched in at least one target application; the search result image is obtained by searching for an accessibility service corresponding to the content to be searched and the target application; performing content recognition processing on the search result image in combination with the big model to obtain recognized content corresponding to the search result image; performing fusion processing on the search result image and the recognized content corresponding to the search result image in combination with the big model to obtain a target search result corresponding to the content to be searched.

[0006] According to another aspect of the present disclosure, a content search device based on big model technology is provided, the device comprising: a first acquisition module, used to acquire content to be searched and a search result image related to the content to be searched in at least one target application; the search result image is obtained by searching for an accessibility service corresponding to the content to be searched and the target application; a second acquisition module, used to perform content recognition processing on the search result image in combination with the big model, and acquire recognition content corresponding to the search result image; and a fusion processing module, used to perform fusion processing on the search result image and the recognition content corresponding to the search result image in combination with the big model, and acquire a target search result corresponding to the content to be searched.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the content search method based on the big model technology proposed above in the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the content search method based on the large model technology proposed above in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the content search method based on the large model technology proposed above in the present disclosure.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 It is a block diagram of an electronic device used to implement the content search method based on the big model technology according to the embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] The current content search method mainly includes: for the content to be searched, the content to be searched can be input into the search box in each target application to obtain the search result image related to the content to be searched in each target application; and the search result image is used as the target search result.

[0019] In the above scheme, there are a large number of search result images, and there may be content overlap and disordered arrangement between multiple search result images, making it difficult to effectively extract required content from the multiple search result images, thereby reducing content search efficiency.

[0020] In view of the above problems, the present disclosure proposes a content search method, device and electronic device based on big model technology.

[0021] Figure 1 It is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the content search method based on the big model technology of the embodiment of the present disclosure can be applied to the content search device based on the big model technology, and the device can be configured in an electronic device so that the electronic device can perform the content search function based on the big model technology. In the following embodiments, the execution subject is an electronic device as an example for explanation.

[0022] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a smart speaker, a server, a server cluster, and other hardware devices with various operating systems, touch screens and / or display screens.

[0023] The content search device based on the big model technology may also be software in an electronic device, such as content search software, etc. In the following embodiments, the content search device based on the big model technology is described as an electronic device.

[0024] like Figure 1 As shown, the content search method based on the big model technology may include the following steps:

[0025] Step 101, obtaining the content to be searched and a search result image related to the content to be searched in at least one target application; the search result image is obtained by searching for the accessibility service corresponding to the content to be searched and the target application.

[0026] In an embodiment of the present disclosure, in one example, the process of the electronic device executing step 101 may be, for example, communicating with each target application, passing the content to be searched to the target application, and obtaining a search result image returned by the target application.

[0027] In another example, multiple services may be provided in the electronic device, and the multiple services may implement the content search function based on the large model technology through the calling relationship. For example, the electronic device may be provided with access service, agent service (intelligent agent service), simulator service, barrier-free service, mongo service, etc.

[0028] Among them, the process of the electronic device executing step 101 in combination with the above-mentioned service can be, for example, accessing the service to receive a search request; carrying the content to be searched in the search request; the accessing service calling the agent list in the mongo service to obtain any agent identifier therein; calling the agent service corresponding to the agent identifier based on the search request; the agent service calling the record in the mongo service to obtain the simulator identifier that is bound to the agent representation and is in an idle state; querying the list in the agent service according to the simulator identifier to obtain the accessibility service corresponding to the simulator identifier; the agent service calling the accessibility service to obtain the search result image.

[0029] Step 102: Perform content recognition processing on the search result image in combination with the large model to obtain the recognized content corresponding to the search result image.

[0030] In an embodiment of the present disclosure, in one example, the process of the electronic device executing step 102 may be, for example, inputting the search result image into a large model to obtain the recognition content output by the large model.

[0031] In another example, the process of the electronic device executing step 102 may be, for example, determining, for each target application, a prompt word corresponding to the target application; the prompt word indicates the content that needs to be extracted from the search result image in the target application; the search result image and the prompt word in the target application are input into the large model to obtain the recognition content corresponding to the search result image.

[0032] It should be noted that the big model can be provided in the form of a big model service. Correspondingly, the electronic device can call the big model service based on the search result image and the prompt word to obtain the recognition content returned by the big model service.

[0033] Different target applications correspond to different prompt words. Take the target applications as the first application and the second application as an example, wherein the first application may be an application for posting. The prompt word corresponding to the first application may indicate to extract at least one of the following contents in the search result image: content related to post comments, content related to the post publishing time, and content related to the post publisher. The second application may be an application for posting videos. The prompt word corresponding to the second application may indicate to extract at least one of the following contents in the search result image: content related to video comments, content related to the video publishing time, and content related to the video publisher.

[0034] Among them, combined with the prompt words corresponding to the target application, the content in the search result image in the target application can be accurately extracted, thereby improving the extraction accuracy of the recognized content.

[0035] Step 103 , combining the large model to fuse the search result image and the recognition content corresponding to the search result image, and obtain the target search result corresponding to the content to be searched.

[0036] In the disclosed embodiment, the process of the electronic device performing step 103 may be, for example, inputting each search result image and the recognition content corresponding to the search result image into the large model, and obtaining the target search result output by the large model. The target search result may be presented in a multimedia page, for example. The multimedia page may include at least one of text, image, video, link, etc.

[0037] It should be noted that, when the big model is provided in the form of a big model service, the electronic device can call the big model service based on each search result image and the corresponding recognition content to obtain the recognition content returned by the big model service.

[0038] In an embodiment of the present disclosure, in order to avoid subsequent repeated search processing of the content to be searched and improve the efficiency of content search, the electronic device can store the content to be searched and the target search results corresponding to the content to be searched in a first database; when the same content to be searched is obtained again, the stored content is queried to obtain the target search results corresponding to the content to be searched.

[0039] The content search method based on the big model technology of the embodiment of the present disclosure obtains the content to be searched and the search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application; the search result image is processed for content recognition in combination with the big model to obtain the recognition content corresponding to the search result image; the search result image and the recognition content corresponding to the search result image are fused in combination with the big model to obtain the target search result corresponding to the content to be searched; wherein the target search result is obtained by fusion of the search result image and the corresponding recognition content, without content overlap and disordered arrangement, and can effectively extract the required content from the target search result, thereby improving the content search efficiency.

[0040] In order to further improve the quality of the target search results determined, the electronic device can combine the scores of the search result images and select candidate search result images with higher scores for fusion processing. Figure 2 As shown, Figure 2 is a schematic diagram according to a second embodiment of the present disclosure, Figure 2 The illustrated embodiment may include the following steps:

[0041] Step 201, obtaining the content to be searched and a search result image related to the content to be searched in at least one target application; the search result image is obtained by searching for the accessibility service corresponding to the content to be searched and the target application.

[0042] Step 202: Perform content recognition processing on the search result image in combination with the large model to obtain the recognized content corresponding to the search result image.

[0043] Step 203 : for each target application, determine the score of the search result image in combination with the recognition content corresponding to the search result image in the target application.

[0044] In the disclosed embodiment, the identified content may include at least one of the following: release time content, comment related content, and publisher related content. Correspondingly, the process of the electronic device executing step 203 may be, for example, to determine at least one of the release time rating value, comment rating value, and publisher rating value in combination with the identified content corresponding to the search result image; the release time rating value is determined based on the release time content; the comment rating value is determined based on the comment related content; the publisher rating value is determined based on the publisher related content; and the rating value corresponding to the search result image is determined based on at least one of the release time rating value, comment rating value, and publisher rating value.

[0045] The electronic device may perform sum processing on at least one of the publishing time rating value, the comment rating value, and the publisher rating value to obtain a rating value corresponding to the search result image.

[0046] For example, assuming that the target application is an application for posting; the search result image can be a screenshot of the page where the post is located in the target application. Correspondingly, the posting time content may include at least one of the following: posting time. Comment-related content may include at least one of the following: number of post likes, number of post collections, number of post comments, number of likes for the top three comments. Publisher-related content may include at least one of the following: number of publisher fans, number of publisher likes.

[0047] Among them, based on the above assumptions, the process of determining the release time score value can be as follows: obtain the acquisition time of the search result image; determine the difference between the acquisition time and the post time; determine the number of days based on the difference; and obtain the release time score value by (number of days - 3) × (-1).

[0048] Based on the above assumptions, the comment score can be determined based on the following formula: number of post likes × 1 + number of post comments × 2 + number of post favorites × 4 + number of top three post likes / 3 × 0.5. Based on the above assumptions, the publisher score can be determined based on the following formula: number of publisher fans / 10,000 + number of publisher likes / 100,000.

[0049] Based on the above assumptions, the publishing time rating value, the comment rating value and the publisher rating value may be added together to obtain the rating value corresponding to the search result image.

[0050] Among them, by combining the release time rating value, the comment rating value and at least one of the publisher rating values, the rating value corresponding to the search result image is determined, and various contents in the recognition content of the search result image can be considered, thereby improving the accuracy of the determined rating value and further improving the quality of the determined target search results.

[0051] Step 204: sort the search result images in the target application according to the score values ​​to obtain a sorting result.

[0052] In the disclosed embodiment, the electronic device may sort the search result images in the target application in descending order according to the score values ​​to obtain a sorting result.

[0053] Step 205: determine the first number of search result images ranked at the top in the sorting result as candidate search result images.

[0054] It should be noted that a scoring service may be provided in the electronic device. Accordingly, the electronic device may call the scoring service based on the search result image and the corresponding recognition content to obtain the candidate search result image returned by the scoring service.

[0055] Step 206 , combining the large model with the candidate search result image in at least one target application and the identification content corresponding to the candidate search result image, to obtain a target search result corresponding to the content to be searched.

[0056] In the disclosed embodiment, the process of the electronic device executing step 206 may, for example, be to input a candidate search result image in at least one target application and the recognition content corresponding to the candidate search result image into the large model to obtain the target search result output by the large model.

[0057] It should be noted that the details of step 201 to step 202 can be found in Figure 1 Steps 101 to 102 in the illustrated embodiment will not be described in detail herein.

[0058] The content search method based on the big model technology of the embodiment of the present disclosure is as follows: obtaining the content to be searched and the search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application; performing content recognition processing on the search result image in combination with the big model to obtain the recognition content corresponding to the search result image; for each target application, determining the score value of the search result image in combination with the recognition content corresponding to the search result image in the target application; sorting the search result images in the target application according to the score value to obtain the sorting result; determining the first number of search result images ranked in the front in the sorting result as candidate search result images; fusing the candidate search result images in at least one target application and the recognition content corresponding to the candidate search result images in combination with the big model to obtain the target search result corresponding to the content to be searched; wherein the target search result is obtained by fusing the candidate search result images with higher scoring values ​​and the corresponding recognition content, which can ensure the quality of the recognition content, thereby improving the quality of the target search result, so that the required content can be extracted from the target search result more effectively, further improving the content search efficiency.

[0059] The correspondence between historical search content and historical search results can be stored in the first database. When obtaining the content to be searched, the first database is queried first, thereby reducing the amount of search result image processing, shortening the content search time, and further improving the content search efficiency. Figure 3 As shown, Figure 3 is a schematic diagram according to a third embodiment of the present disclosure, Figure 3 The illustrated embodiment may include the following steps:

[0060] Step 301, obtaining the content to be searched.

[0061] Step 302: Determine whether there is target historical search content matching the content to be searched in the first database; the first database includes a correspondence between historical search content and historical search results.

[0062] In the embodiment of the present disclosure, a cache service may be provided in the electronic device. The electronic device may call the cache service based on the content to be searched, and obtain the target historical search content and the historical search results corresponding to the target historical search content returned by the cache service.

[0063] In an embodiment of the present disclosure, in one example, the first database also includes historical content vectors of historical search content. The process of the electronic device executing step 302 may, for example, include determining a search content vector of the content to be searched; determining the vector similarity between the search content vector and each historical content vector in the first database; if the first historical search content exists in the first database, determining the first historical search content as the target historical search content; and the vector similarity between the historical content vector of the first historical search content and the search content vector is greater than or equal to a first similarity threshold.

[0064] The cache service in the electronic device may call the vectorization service (embedding service) based on the content to be searched, and obtain the search content vector returned by the vectorization service. In addition, the cache service may also call the vectorization service based on the historical search content, obtain the historical content vector returned by the vectorization service, and store the historical content vector in a record including the historical search content in the first database.

[0065] Among them, combining the vector similarity between the historical content vector of the historical search content and the search content vector of the content to be searched, selecting the first historical search content from various historical search contents as the target historical search content can improve the matching degree between the target historical search content and the content to be searched, and further improve the matching degree between the target search results and the content to be searched.

[0066] In another example of the disclosed embodiment, the first database also includes historical content vectors of historical search content. Correspondingly, the process of the electronic device executing step 302 may be, for example, determining whether there is a second historical search content in the first database; the vector similarity between the historical content vector of the second historical search content and the search content vector of the content to be searched is less than the first similarity threshold and greater than or equal to the second similarity threshold; if the second historical search content exists in the first database, determining the search segmentation terms of the content to be searched and the historical segmentation terms of the second historical search content; determining the proportion of the same segmentation terms in the search segmentation terms and the historical segmentation terms; if the proportion is greater than or equal to the proportion threshold, determining the second historical search content as the target historical search content.

[0067] Among them, the process of determining the proportion of identical participle words in search participle words and historical participle words can, for example, be to determine the number of identical participle words in search participle words and historical participle words; determine the maximum value of the number of search participle words and the number of historical participle words; and determine the ratio of the number of identical participle words to the maximum value as the proportion of identical participle words.

[0068] Among them, the cache service in the electronic device can call the segmentation service based on the content to be searched and obtain the search segmentation terms returned by the segmentation service; the cache service can call the segmentation service based on the second historical search content and obtain the historical segmentation terms returned by the segmentation service.

[0069] Among them, combining the vector similarity between the historical content vector of the historical search content and the search content vector of the content to be searched, as well as the proportion of the same segmentation terms in the historical segmentation terms of the historical search content and the search segmentation terms of the content to be searched, to select the target historical search content, can further improve the matching degree between the target historical search content and the content to be searched, and further improve the matching degree between the target search results and the content to be searched.

[0070] In another example of the disclosed embodiment, the first database also includes rewritten historical search content corresponding to the historical search content. Correspondingly, the process of the electronic device executing step 302 may be, for example, determining the rewritten content to be searched corresponding to the content to be searched; determining whether there is a third historical search content in the first database; the vector similarity between the historical content vector of the rewritten historical search content corresponding to the third historical search content and the search content vector of the rewritten content to be searched is greater than or equal to the first similarity threshold; if there is a third historical search content, determining the third historical search content as the target historical search content.

[0071] The cache service in the electronic device may call the vectorization service based on rewriting the content to be searched, and obtain the search content vector returned by the vectorization service. The cache service may also call the vectorization service based on rewriting the historical search content, and obtain the historical content vector returned by the vectorization service.

[0072] The cache service in the electronic device can call the big model service based on the historical search content to obtain the rewritten historical search content returned by the big model service. The cache service can also call the big model service based on the content to be searched to obtain the rewritten content to be searched returned by the big model service.

[0073] Among them, combining the vector similarity between the historical content vector that rewrites the historical search content and the search content vector that rewrites the content to be searched to select the target historical search content can increase the probability of selecting the target historical search content, thereby further reducing the amount of search result image processing and further improving content search efficiency.

[0074] It should be noted that the electronic device can jointly determine the target historical search content based on two or three of the above three examples. For example, the electronic device can determine the target historical search content based on the first example; if the target historical search content is not determined, the target historical search content is determined based on the second example. The order of the first example and the second example can be adjusted, and the target historical search content can be determined based on the second example first; if the target historical search content is not determined, the target historical search content is determined based on the first example.

[0075] For another example, the electronic device may determine the target historical search content based on the first example; if the target historical search content is not determined, the electronic device may determine the target historical search content based on the second example; if the target historical search content is not determined, the electronic device may determine the target historical search content based on the third example. The order of the first example, the second example, and the third example may be adjusted.

[0076] For another example, the electronic device may determine the target historical search content based on the second example; if the target historical search content is not determined, the electronic device may determine the target historical search content based on the third example, wherein the order between the second example and the third example may be adjusted.

[0077] Step 303: When the target historical search content exists in the first database, the historical search result corresponding to the target historical search content in the first database is determined as the target search result corresponding to the content to be searched.

[0078] Step 304, when the target historical search content does not exist in the first database, obtain a search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application.

[0079] Step 305 , performing content recognition processing on the search result image in combination with the large model to obtain the recognized content corresponding to the search result image.

[0080] Step 306 , combining the large model to fuse the search result image and the recognition content corresponding to the search result image, to obtain the target search result corresponding to the content to be searched.

[0081] It should be noted that the details of step 301, step 304 to step 306 can be found in Figure 1 Steps 101 to 103 in the illustrated embodiment will not be described in detail herein.

[0082] The content search method based on the big model technology of the embodiment of the present disclosure is as follows: obtaining the content to be searched; determining whether there is target historical search content matching the content to be searched in the first database; including the correspondence between the historical search content and the historical search results in the first database; in case the target historical search content exists in the first database, determining the historical search results corresponding to the target historical search content in the first database as the target search results corresponding to the content to be searched; in case the target historical search content does not exist in the first database, obtaining the search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application; performing content recognition processing on the search result image in combination with the big model to obtain the recognition content corresponding to the search result image; performing fusion processing on the search result image and the recognition content corresponding to the search result image in combination with the big model to obtain the target search result corresponding to the content to be searched; wherein the correspondence between the historical search content and the historical search results is stored in the first database, and when the content to be searched is obtained, the first database is queried first, thereby reducing the processing amount of the search result image, shortening the content search time, and further improving the content search efficiency.

[0083] In order to implement the above embodiment, the present disclosure also provides a content search device based on large model technology. Figure 4 As shown, Figure 4 4 is a schematic diagram according to the fourth embodiment of the present disclosure. The content search device 40 based on the large model technology may include: a first acquisition module 401 , a second acquisition module 402 and a fusion processing module 403 .

[0084] Among them, the first acquisition module 401 is used to obtain the content to be searched and the search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application; the second acquisition module 402 is used to perform content recognition processing on the search result image in combination with the large model to obtain the recognized content corresponding to the search result image; the fusion processing module 403 is used to perform fusion processing on the search result image and the recognized content corresponding to the search result image in combination with the large model to obtain the target search result corresponding to the content to be searched.

[0085] As a possible implementation method of the embodiment of the present disclosure, the second acquisition module 402 is specifically used to, for each target application, determine a prompt word corresponding to the target application; the prompt word indicates the content that needs to be extracted from the search result image in the target application; the search result image in the target application and the prompt word are input into the large model to obtain the recognition content corresponding to the search result image.

[0086] As a possible implementation method of an embodiment of the present disclosure, the fusion processing module 403 includes a first determination unit, a sorting processing unit, a second determination unit and a fusion processing unit; the first determination unit is used to determine, for each target application, a score value of the search result image in combination with the recognition content corresponding to the search result image in the target application; the sorting processing unit is used to sort the search result images in the target application according to the score value to obtain a sorting result; the second determination unit is used to determine a first number of search result images ranked at the front in the sorting result as candidate search result images; the fusion processing unit is used to fuse the candidate search result images in the at least one target application and the recognition content corresponding to the candidate search result image in combination with the large model to obtain the target search result.

[0087] As a possible implementation method of an embodiment of the present disclosure, the identified content includes at least one of the following: release time content, comment-related content, and publisher-related content; the first determination unit is specifically used to determine at least one of a release time rating value, a comment rating value, and a publisher rating value in combination with the identified content corresponding to the search result image; the release time rating value is determined based on the release time content; the comment rating value is determined based on the comment-related content; the publisher rating value is determined based on the publisher-related content; and the rating value corresponding to the search result image is determined based on at least one of the release time rating value, the comment rating value, and the publisher rating value.

[0088] As a possible implementation manner of the embodiment of the present disclosure, the device further includes: a storage module, configured to store the content to be searched and the target search results corresponding to the content to be searched in a first database.

[0089] As a possible implementation method of the embodiment of the present disclosure, the device also includes: a first determination module and a second determination module; the first determination module is used to determine whether there is target historical search content matching the content to be searched in the first database; the first database includes a correspondence between historical search content and historical search results; the second determination module is used to, when the target historical search content exists in the first database, determine the historical search results corresponding to the target historical search content in the first database as the target search results corresponding to the content to be searched.

[0090] As a possible implementation method of an embodiment of the present disclosure, the first database also includes historical content vectors of the historical search content; the first determination module is specifically used to determine the search content vector of the content to be searched; determine the vector similarity between the search content vector and each historical content vector in the first database; in the case that there is a first historical search content in the first database, determine the first historical search content as the target historical search content; the vector similarity between the historical content vector of the first historical search content and the search content vector is greater than or equal to a first similarity threshold.

[0091] As a possible implementation method of the embodiment of the present disclosure, the first database also includes historical content vectors of the historical search content; the first determination module is specifically used to determine whether there is a second historical search content in the first database; the vector similarity between the historical content vector of the second historical search content and the search content vector of the content to be searched is less than a first similarity threshold and greater than or equal to a second similarity threshold; in the case that the second historical search content exists in the first database, determine the search segmentation terms of the content to be searched and the historical segmentation terms of the second historical search content; determine the proportion of the same segmentation terms in the search segmentation terms and the historical segmentation terms; in the case that the proportion is greater than or equal to the proportion threshold, determine the second historical search content as the target historical search content.

[0092] As a possible implementation method of the embodiment of the present disclosure, the first database also includes rewritten historical search content corresponding to the historical search content; the rewritten historical search content is determined in combination with the historical search content and the large model; the first determination module is specifically used to determine the rewritten content to be searched corresponding to the content to be searched; determine whether there is a third historical search content in the first database; the vector similarity between the historical content vector of the rewritten historical search content corresponding to the third historical search content and the search content vector of the rewritten content to be searched is greater than or equal to a first similarity threshold; in the case that the third historical search content exists, the third historical search content is determined as the target historical search content.

[0093] The content search device based on the big model technology in the embodiment of the present disclosure obtains the content to be searched and the search result image related to the content to be searched in at least one target application; the search result image is obtained by searching the accessibility service corresponding to the content to be searched and the target application; the search result image is processed for content recognition in combination with the big model to obtain the recognition content corresponding to the search result image; the search result image and the recognition content corresponding to the search result image are fused in combination with the big model to obtain the target search result corresponding to the content to be searched; wherein the target search result is obtained by fusion of the search result image and the corresponding recognition content, without content overlap and disordered arrangement, and can effectively extract the required content from the target search result, thereby improving the content search efficiency.

[0094] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, comply with the relevant laws and regulations, and do not violate public order and good morals.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0096] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0097] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as a content search method based on a large model technology. For example, in some embodiments, a content search method based on a large model technology may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the content search method based on the large model technology described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the content search method based on the large model technology in any other appropriate manner (for example, by means of firmware).

[0100] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0102] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0105] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0107] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A content search method based on a large model technology, the method comprising: Acquire content to be searched and a search result image related to the content to be searched in at least one target application; The search result image is obtained by searching in combination with the content to be searched and the accessibility service corresponding to the target application; Performing content recognition processing on the search result image in combination with the large model to obtain recognition content corresponding to the search result image; The search result image and the identification content corresponding to the search result image are fused in combination with the large model to obtain a target search result corresponding to the content to be searched.

2. The method according to claim 1, wherein: The step of performing content recognition processing on the search result image in combination with the large model to obtain the recognition content corresponding to the search result image includes: For each target application, determining a prompt word corresponding to the target application; the prompt word indicates the content to be extracted from the search result image in the target application; The search result image in the target application and the prompt word are input into the large model to obtain the recognition content corresponding to the search result image.

3. The method according to claim 1, wherein: The step of fusing the search result image and the identification content corresponding to the search result image in combination with the large model to obtain a target search result corresponding to the content to be searched includes: For each target application, determining a score value of the search result image in combination with the recognition content corresponding to the search result image in the target application; Sorting the search result images in the target application according to the scoring values ​​to obtain a sorting result; Determine a first number of search result images ranked at the top in the ranking result as candidate search result images; The candidate search result image in the at least one target application and the recognition content corresponding to the candidate search result image are fused in combination with the large model to obtain the target search result.

4. The method according to claim 3, wherein: The identified content includes at least one of the following: publishing time content, comment related content, publisher related content; The step of determining the score value of the search result image in combination with the recognition content corresponding to the search result image in the target application includes: Determine at least one of a publishing time rating value, a comment rating value, and a publisher rating value in combination with the identification content corresponding to the search result image; the publishing time rating value is determined based on the publishing time content; the comment rating value is determined based on the comment-related content; and the publisher rating value is determined based on the publisher-related content; The scoring value corresponding to the search result image is determined according to at least one of the publishing time scoring value, the comment scoring value, and the publisher scoring value.

5. The method according to claim 1, wherein: The method further comprises: The content to be searched and the target search results corresponding to the content to be searched are stored in a first database.

6. The method according to claim 1, wherein: Before acquiring a search result image related to the to-be-searched content in at least one target application, the method further includes: Determine whether there is a target historical search content matching the content to be searched in the first database; the first database includes a correspondence between the historical search content and the historical search results; In a case where the target historical search content exists in the first database, the historical search result corresponding to the target historical search content in the first database is determined as the target search result corresponding to the content to be searched.

7. The method according to claim 6, wherein: The first database also includes a historical content vector of the historical search content; and determining whether there is a target historical search content matching the content to be searched in the first database includes: Determining a search content vector of the content to be searched; Determining vector similarities between the search content vector and each historical content vector in the first database; When there is a first historical search content in the first database, the first historical search content is determined as the target historical search content; and the vector similarity between the historical content vector of the first historical search content and the search content vector is greater than or equal to a first similarity threshold.

8. The method according to claim 6 or 7, wherein: The first database also includes a historical content vector of the historical search content; and determining whether there is a target historical search content matching the content to be searched in the first database includes: Determine whether there is a second historical search content in the first database; the vector similarity between the historical content vector of the second historical search content and the search content vector of the content to be searched is less than a first similarity threshold and greater than or equal to a second similarity threshold; In the case where the second historical search content exists in the first database, determining the search segmentation terms of the content to be searched and the historical segmentation terms of the second historical search content; Determine the proportion of identical segmentation terms in the search segmentation terms and the historical segmentation terms; When the proportion is greater than or equal to a proportion threshold, the second historical search content is determined as the target historical search content.

9. The method according to claim 6 or 7, wherein: The first database also includes rewritten historical search content corresponding to the historical search content; the rewritten historical search content is determined by combining the historical search content and the large model; The step of determining whether there is target historical search content matching the content to be searched in the first database includes: Determine the rewritten content to be searched corresponding to the content to be searched; Determine whether there is a third historical search content in the first database; the vector similarity between the historical content vector of the rewritten historical search content corresponding to the third historical search content and the search content vector of the rewritten content to be searched is greater than or equal to a first similarity threshold; In a case where the third history search content exists, the third history search content is determined as the target history search content.

10. A content search device based on large model technology, the device comprising: A first acquisition module, configured to acquire content to be searched and a search result image related to the content to be searched in at least one target application; The search result image is obtained by searching in combination with the content to be searched and the accessibility service corresponding to the target application; A second acquisition module is used to perform content recognition processing on the search result image in combination with the large model to obtain the recognition content corresponding to the search result image; A fusion processing module is used to combine the large model to perform fusion processing on the search result image and the identification content corresponding to the search result image to obtain a target search result corresponding to the content to be searched.

11. The device according to claim 10, wherein: The second acquisition module is specifically used to: For each target application, determining a prompt word corresponding to the target application; the prompt word indicates the content to be extracted from the search result image in the target application; The search result image in the target application and the prompt word are input into the large model to obtain the recognition content corresponding to the search result image.

12. The device according to claim 10, wherein: The fusion processing module includes a first determination unit, a sorting processing unit, a second determination unit and a fusion processing unit; The first determination unit is used to determine, for each target application, a score value of the search result image in combination with the identification content corresponding to the search result image in the target application; The sorting processing unit is used to sort the search result images in the target application according to the scoring value to obtain a sorting result; The second determining unit is used to determine a first number of search result images ranked at the top in the ranking result as candidate search result images; The fusion processing unit is used to perform fusion processing on the candidate search result image in the at least one target application and the recognition content corresponding to the candidate search result image in combination with the large model to obtain the target search result.

13. The device according to claim 12, wherein: The identified content includes at least one of the following: publishing time content, comment related content, publisher related content; the first determination unit is specifically used to: Determine at least one of a publishing time rating value, a comment rating value, and a publisher rating value in combination with the identification content corresponding to the search result image; the publishing time rating value is determined based on the publishing time content; the comment rating value is determined based on the comment-related content; and the publisher rating value is determined based on the publisher-related content; The scoring value corresponding to the search result image is determined according to at least one of the publishing time scoring value, the comment scoring value, and the publisher scoring value.

14. The device according to claim 10, wherein: The device further includes: a storage module, configured to store the content to be searched and the target search results corresponding to the content to be searched in a first database.

15. The device according to claim 10, wherein: The device further includes: a first determining module and a second determining module; The first determination module is used to determine whether there is a target historical search content matching the content to be searched in the first database; the first database includes a correspondence between the historical search content and the historical search results; The second determination module is used to determine the historical search results corresponding to the target historical search content in the first database as the target search results corresponding to the content to be searched when the target historical search content exists in the first database.

16. The device according to claim 15, wherein: The first database also includes the historical content vector of the historical search content; the first determination module is specifically used to: Determining a search content vector of the content to be searched; Determining vector similarities between the search content vector and each historical content vector in the first database; When there is a first historical search content in the first database, the first historical search content is determined as the target historical search content; and the vector similarity between the historical content vector of the first historical search content and the search content vector is greater than or equal to a first similarity threshold.

17. The device according to claim 15 or 16, wherein: The first database also includes the historical content vector of the historical search content; the first determination module is specifically used to: Determine whether there is a second historical search content in the first database; the vector similarity between the historical content vector of the second historical search content and the search content vector of the content to be searched is less than a first similarity threshold and greater than or equal to a second similarity threshold; In the case where the second historical search content exists in the first database, determining the search segmentation terms of the content to be searched and the historical segmentation terms of the second historical search content; Determine the proportion of identical segmentation terms in the search segmentation terms and the historical segmentation terms; When the proportion is greater than or equal to a proportion threshold, the second historical search content is determined as the target historical search content.

18. The device according to claim 15 or 16, wherein: The first database also includes rewritten historical search content corresponding to the historical search content; the rewritten historical search content is determined by combining the historical search content and the large model; the first determination module is specifically used to: Determine the rewritten content to be searched corresponding to the content to be searched; Determine whether there is a third historical search content in the first database; the vector similarity between the historical content vector of the rewritten historical search content corresponding to the third historical search content and the search content vector of the rewritten content to be searched is greater than or equal to a first similarity threshold; In a case where the third history search content exists, the third history search content is determined as the target history search content.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images