Vehicle information query method and device, electronic equipment and storage medium

Through the vehicle information query method, image retrieval and semantic analysis are used to generate query results, solving the problem of low efficiency in vehicle information query, achieving efficient and accurate information acquisition, and improving user experience.

CN118916508APending Publication Date: 2024-11-08CHONGQING WUTONG CAR LINK TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411058511.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Vehicle information query is inefficient, and users need to manually search the detailed vehicle instructions to find specific operation or troubleshooting information.

Method used

Image search based on vehicle problem query information, semantic analysis combined with text slices, intersection processing, and query results are generated to avoid users manually flipping the manual.

Benefits of technology

It improves the efficiency and accuracy of vehicle information query, so users do not need to manually find relevant text or image information. The query results include dual outputs of images and text, improving the reading experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916508A_ABST
    Figure CN118916508A_ABST
Patent Text Reader

Abstract

The invention relates to a vehicle information query method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the image retrieval according to the problem query information of a vehicle, obtaining first alternative image information, carrying out the semantic analysis based on the problem query information in combination with a text slice corresponding to the problem query information, and obtaining the first alternative image information; the method comprises the steps of obtaining text answer information, performing image retrieval according to the answer text information to obtain second alternative image information, performing intersection processing based on the first alternative image information and the second alternative image information to obtain target image information, and generating a query result corresponding to question query information based on the answer text information and the target image information. Therefore, the user does not need to manually check the vehicle specification, the problem that the vehicle information query efficiency is low when the user manually checks the vehicle specification is solved, and the vehicle information query efficiency can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a vehicle information query method, device, electronic equipment and storage medium. Background Art

[0002] With the development of society, vehicles have gradually become an indispensable means of transportation in daily life, and the speed of vehicle updates and iterations is also accelerating, which has led to a gradual increase in the types and functions of vehicles. For users, during the use of vehicles, they often encounter functional failures of the vehicles, or need to query the functional operations of the vehicles. In this scenario, users need to query the manual of the vehicle, and the manual of the vehicle can be a paper manual or an electronic manual. Regardless of whether it is a paper manual or an electronic manual, users are required to manually check the contents of the manual until they find the vehicle operating instructions they need to query, resulting in low efficiency in vehicle information query. Summary of the invention

[0003] The present application provides a vehicle information query method, device, electronic device and storage medium to solve the problem of low efficiency of vehicle information query.

[0004] In a first aspect, the present application provides a vehicle information query method, comprising:

[0005] Perform image retrieval based on the problem query information of the vehicle to obtain first candidate image information;

[0006] Based on the question query information, semantic analysis is performed in combination with the text slice corresponding to the question query information to obtain text answer information;

[0007] Performing image retrieval based on the answer text information to obtain second candidate image information;

[0008] Performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information;

[0009] Based on the answer text information and the target image information, a query result corresponding to the question query information is generated.

[0010] Optionally, performing image retrieval based on the problem query information of the vehicle to obtain the first candidate image information includes:

[0011] Perform image retrieval based on the question query information to obtain first image information;

[0012] Using the question query information to perform text retrieval to obtain the text slice;

[0013] The text slice corresponds to preset image information, and is determined as second image information;

[0014] The first candidate image information is obtained by performing intersection processing based on the first image information and the second image information.

[0015] Optionally, the using the question query information to perform text retrieval to obtain the text slice includes:

[0016] Vectorizing the question query information to obtain a question query vector;

[0017] Comparing the question query vector with a preset slice vector to obtain a target slice vector that matches the question query vector;

[0018] The text slice is generated by using the slice information corresponding to the target slice vector.

[0019] Optionally, performing intersection processing based on the first image information and the second image information to obtain the first candidate image information includes:

[0020] Acquire at least one first image corresponding to the first image information, and acquire at least one second image corresponding to the second image information;

[0021] Compare each of the first images with each of the second images to obtain a first comparison result set, wherein the first comparison result set includes at least one first comparison result;

[0022] When the first comparison result is an identical image result, determining the first image or the second image corresponding to the first comparison result as a first candidate image;

[0023] The first candidate image is used to generate the first candidate image information.

[0024] Optionally, performing image retrieval based on the question query information to obtain first image information includes:

[0025] Vectorizing the question query information to obtain a question query vector;

[0026] Get the preset image description vector;

[0027] Comparing the question query vector with the image description vector to obtain a target image description vector that matches the question query vector;

[0028] The image information corresponding to the target image description vector is determined as the first image information.

[0029] Optionally, obtaining the image description vector includes:

[0030] Obtaining vehicle description information corresponding to the vehicle;

[0031] Extracting a vehicle description image from the vehicle description information, and obtaining an image description text corresponding to the vehicle description image;

[0032] The image description text is vectorized to obtain the image description vector corresponding to the vehicle description image.

[0033] Optionally, performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information includes:

[0034] Acquire at least one third image corresponding to the first candidate image information, and acquire at least one fourth image corresponding to the second candidate image information;

[0035] Comparing each of the third images with each of the fourth images to obtain a second comparison result set, wherein the second comparison result set includes at least one second comparison result;

[0036] When the second comparison result is an identical image result, determining the third image or the fourth image corresponding to the second comparison result as a second candidate image;

[0037] The target image information is generated using the second candidate image.

[0038] Optionally, before performing image retrieval based on the problem query information of the vehicle, the method further includes:

[0039] Obtaining vehicle description information corresponding to the vehicle;

[0040] extracting a vehicle description text from the vehicle description information;

[0041] The vehicle description text is sliced ​​to obtain the text slices.

[0042] Optionally, the performing semantic analysis based on the question query information and in combination with a text slice corresponding to the question query information to obtain text answer information includes:

[0043] Generate prompt information according to the question query information and the text slice;

[0044] Inputting the prompt information into a preset semantic analysis model to obtain a model output result, wherein the model output result represents the text output by the semantic analysis model;

[0045] Based on the model output result, the text answer information is determined.

[0046] In a second aspect, the present application provides a vehicle information query device, comprising:

[0047] A first image retrieval module, used to perform image retrieval based on the problem query information of the vehicle to obtain first candidate image information;

[0048] A semantic analysis module, used to perform semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information;

[0049] A second image retrieval module, used to perform image retrieval based on the answer text information to obtain second candidate image information;

[0050] An intersection processing module, used for performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information;

[0051] A generation module is used to generate a query result corresponding to the question query information based on the answer text information and the target image information.

[0052] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0053] Memory, used to store computer programs;

[0054] The processor is used to implement the vehicle information query method described in any one of the first aspects when executing the program stored in the memory.

[0055] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the vehicle information query method as described in any one of the first aspects is implemented.

[0056] The embodiment of the present application obtains first candidate image information by performing image retrieval based on the question query information of the vehicle, performs semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information, and performs image retrieval based on the answer text information to obtain second candidate image information, then performs intersection processing based on the first candidate image information and the second candidate image information to obtain target image information, and generates a query result corresponding to the question query information based on the answer text information and the target image information, thereby eliminating the need for the user to manually check the vehicle manual, solving the problem of low efficiency of vehicle information query caused by manually checking the vehicle manual, and effectively improving the efficiency of vehicle information query. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a process flow of a vehicle information query method provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of an application scenario of a vehicle information query method provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of the structure of a vehicle information query device provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0062] Nowadays, the functions of vehicles are constantly increasing, making it difficult for users to fully understand the various functions of the vehicle when using the vehicle, and they are also unable to directly determine the corresponding handling methods when the vehicle breaks down or has an abnormality, which brings great difficulties to users in using the vehicle. At this time, the vehicle manual corresponding to the vehicle can be used to inquire about the usage of its various functions or the corresponding handling methods for vehicle failures and abnormalities. However, the vehicle manual usually records in detail the usage methods of various functions and operations of the vehicle, as well as related information on maintenance and troubleshooting, which will make the overall content of the vehicle manual relatively large. As a result, when users inquire about the vehicle manual according to specific needs, they need to spend a lot of time and energy to read and inquire, in order to find the specific content that meets their needs, that is, there is a problem of low efficiency in vehicle information query.

[0063] In order to solve the problem of low efficiency of vehicle information query in the prior art, the present application provides a vehicle information query method, device, electronic device and storage medium, by performing image retrieval based on the vehicle's question query information to obtain first alternative image information, performing semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information, and performing image retrieval based on the answer text information to obtain second alternative image information, then performing intersection processing based on the first alternative image information and the second alternative image information to obtain target image information, and generating a query result corresponding to the question query information based on the answer text information and the target image information, thereby eliminating the need for users to manually check the vehicle manual, solving the problem of low efficiency of vehicle information query caused by manually checking the vehicle manual, and effectively improving the efficiency of vehicle information query.

[0064] Figure 1 A flow chart of a vehicle information query method provided in an embodiment of the present application. This method can be applied to one or more electronic devices such as a vehicle, a client, and a server. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0065] like Figure 1 As shown, a vehicle information query method provided by an embodiment of the present application may specifically include the following steps:

[0066] Step S110: performing image retrieval based on the problem query information of the vehicle to obtain first candidate image information.

[0067] Specifically, the question query information in this embodiment represents information used to query vehicle information, which can be information input by a user according to query requirements, or query information input by a server or client, and this embodiment does not make any specific limitations on this; image retrieval can mean using the question query information to search in a preset database, and determining the retrieved image as the first candidate image information; since image retrieval can obtain one or more images, the first candidate image information can include one or more images, and this embodiment does not make any specific limitations on this.

[0068] In addition, before executing the content of step S110, it is also possible to detect whether the problem query information is received, and after receiving the problem query information, perform image retrieval based on the problem query information of the vehicle to obtain the step content of the first candidate image information.

[0069] Step S120: Based on the question query information, semantic analysis is performed in combination with the text slice corresponding to the question query information to obtain text answer information.

[0070] Specifically, after receiving the question query information, a semantic analysis can be performed based on the question query information and in combination with a text slice corresponding to the question query information, wherein the text slice can be a slice content of the vehicle manual that matches the question query information, and the slice content can be a text content pre-segmented according to the vehicle manual content. The specific segmentation method can be segmentation according to vehicle functions or problems, so that the content of the vehicle manual can be pre-divided into one or more text slices that match the vehicle functions or problems. After receiving the question query information, the present embodiment can determine the slice matching degree between the question query information and each pre-divided text slice, and determine the text slice corresponding to the question query information based on the slice matching degree, which can be a text slice with a slice matching degree greater than a preset matching degree, as the text slice corresponding to the question query information. Furthermore, the question query information can be combined with the text slice corresponding to the question query information to perform a semantic analysis to obtain text answer information, and the text answer information can represent the text answer that matches the question query information in the text content represented by the text slice.

[0071] In one example, when the problem query information obtained is the air conditioner cannot heat the problem query information, it can be determined that the text slice corresponding to the air conditioner cannot heat the problem query information is the air conditioner fault processing text slice, and the air conditioner fault processing text slice may include a variety of air conditioner fault processing related texts, such as air conditioner start-up exception processing, air conditioner shutdown exception processing, air conditioner cooling exception processing, air conditioner heating exception processing, etc. At this time, the air conditioner cannot heat the problem query information and the air conditioner fault processing text slice can be combined with the air conditioner fault processing text slice to perform semantic analysis using a semantic analysis model, and the text answer that matches the air conditioner cannot heat the problem query information is determined in the text content represented by the air conditioner fault processing text slice. At this time, the text answer can be the related text of the air conditioner heating exception processing, thereby achieving the purpose of accurately determining the text answer information corresponding to the problem query information and improving the efficiency of problem query.

[0072] Step S130: performing image retrieval based on the answer text information to obtain second candidate image information.

[0073] Specifically, after obtaining the answer text information, the answer text information can be used to perform image retrieval, where image retrieval can mean using the answer text information to perform a search in a preset database, and determining the retrieved image as the second alternative image information. Since image retrieval can obtain one or more images, the second alternative image information can include one or more images, and this embodiment does not make any specific limitations on this.

[0074] It should be noted that, in the present embodiment, a specific method for performing image retrieval may be to vectorize the answer text information to obtain an answer text vector, wherein the vectorization is to perform a semantic vector conversion, and the answer text in the answer text information may be extracted, and the answer text is converted into a semantic vector to obtain an answer text vector; then the answer text vector may be compared with an image description vector in a preset database to obtain a vector matching degree, and when the vector matching degree meets a preset matching degree threshold, the image corresponding to the image description vector is determined as a second candidate image, and at this time one or more second candidate images may be determined, and a set of second candidate images may be used to generate second candidate image information; wherein, the image description vector in the preset database The quantity represents the semantic vector corresponding to the image description text, and the image description text is the description text used to describe the vehicle description image in the vehicle manual. For example, the vehicle description image is image A, and the image description text can be a text used to represent the content displayed by image A and the corresponding vehicle function examples; the database in this embodiment can be a database pre-established for storing vectors, and the vectors stored in the database can include image description vectors and slice vectors corresponding to text slices. The specific image description vectors and slice vectors can be stored in different database tables; of course, the above is only an example for illustration. In the specific implementation, it can be adaptively adjusted according to the application scenario and requirements, and this embodiment does not make specific limitations on this.

[0075] Step S140: performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information.

[0076] Specifically, since the first alternative image information is obtained by performing image retrieval based on the question query information of the vehicle, and the second alternative image information is obtained by performing image retrieval based on the answer text information, the first alternative image information represents an image that matches the question query information, and the second alternative image information represents an image that matches the answer text information; thus, the first alternative image information and the second alternative image information can be subjected to intersection processing to obtain target image information, wherein the intersection processing represents identifying the intersection image existing in the first alternative image information and the second alternative image information, and the intersection image represents the same image between the first alternative image information and the second alternative image information, and thus the target image information is generated using the intersection image, and the target image information may contain one or more images that are the same between the first alternative image information and the second alternative image information; thus, in this embodiment, the first alternative image information and the second alternative image information are subjected to intersection processing, so that the image represented by the target image information can be an image that is related to the query content required by the question query information, thereby improving the accuracy of vehicle information query.

[0077] Step S150: Generate a query result corresponding to the question query information based on the answer text information and the target image information.

[0078] Specifically, since the answer text information can represent the text answer that matches the question query information, and the target image information can represent the image that is relevant to the query content required by the question query information, the answer text information and the target image information can be combined to generate a query result corresponding to the question query information. The query result can represent output information used to answer the question query information. The output information can include the answer text represented by the answer text information and the target image represented by the target image information, that is, it plays the role of jointly outputting image and text, thereby achieving the purpose of improving the user's reading experience.

[0079] In this embodiment, by performing image retrieval based on the question query information of the vehicle, the first candidate image information is obtained, and semantic analysis is performed based on the question query information and the text slice corresponding to the question query information to obtain text answer information, so that the text answer information matching the question query information can be obtained by performing semantic analysis on the question query information and the text slice, without the need for the user to manually flip through the vehicle manual to find the relevant text; then, image retrieval can be performed based on the answer text information to obtain the second candidate image information, and intersection processing can be performed based on the first candidate image information and the second candidate image information to obtain the target image information, and then the target image information matching the question query information can be obtained by performing intersection processing on the first candidate image information and the second candidate image information, without the need for the user to manually flip through the vehicle manual to find the relevant image; then, based on the answer text information and the target image information, a query result corresponding to the question query information can be generated, so that the user does not need to manually flip through the vehicle manual, and the query result containing the answer text represented by the answer text information and the target image represented by the target image information is adaptively generated, which solves the problem of low efficiency of vehicle information query existing in manually flipping through the vehicle manual, and can also play the role of outputting images and texts together, which can effectively improve the user reading experience and vehicle information query efficiency.

[0080] In the existing related technologies, image retrieval usually only uses the input retrieval information for a single search. At this time, the images retrieved are more random and have a higher probability of meeting the actual needs of the user, that is, there is a problem of low image retrieval accuracy.

[0081] In an optional implementation of the present application, image retrieval is performed based on the problem query information of the vehicle to obtain first alternative image information, which may specifically include the following sub-steps: performing image retrieval based on the problem query information to obtain first image information; performing text retrieval using the problem query information to obtain text slices; corresponding the text slices to preset image information to determine as second image information; performing intersection processing based on the first image information and the second image information to obtain first alternative image information.

[0082] In the present embodiment, in the process of determining the first candidate image information, firstly, an image search may be performed based on the question query information to obtain the first image information matching the question query information, wherein the first image information may include one or more first images; then, the question query information may be used to perform a text search to obtain a text slice, so that the same question query information may be used to perform an image search and a text search respectively to obtain the first image information and the text slice matching the question query information; since the text slice is pre-segmented according to the vehicle manual to obtain the text, and the vehicle manual usually has text and images overlapped for display and explanation, each text slice is segmented according to the vehicle manual to obtain the text slice, and its position in the vehicle manual is determined by the image search and text slice selection. There may be corresponding image information in the manual. For example, text slice A is the content of pages 10 to 12 in the vehicle manual. At this time, the image information corresponding to text slice A is the image contained in pages 10 to 12 in the vehicle manual. After obtaining the text slice, the preset image information corresponding to the text slice can be obtained, and the preset image information corresponding to the text slice can be determined as the second image information, thereby obtaining the second image information matching the text slice. Subsequently, the first image information and the second image information can be intersection-processed to obtain the first alternative image information. The first alternative image information can include one or more first alternative images. The first alternative image is a first image representing that the first image information is identical to the second image information.

[0083] In the present embodiment, in the process of obtaining the first candidate image information, the image retrieval and the text retrieval are respectively performed by using the question query information to obtain the first image information and the text slice matching the question query information, and the text slice is further corresponded to the preset image information to be determined as the second image information, and then the first image information and the second image information are intersection processed to obtain the first candidate image information, which plays the role of performing two image retrievals and image screening using the question query information, in which the image retrieval is performed using the question query information, and the other is to determine the text slice corresponding to the preset image information as the second image information; it avoids the problem of low image retrieval accuracy in the existing related technology of image retrieval, which only uses the input retrieval information for a single retrieval, and can effectively improve the accuracy of image retrieval.

[0084] In an optional embodiment of the present application, question query information is used to perform text retrieval to obtain text slices, which may specifically include the following sub-steps: vectorizing the question query information to obtain a question query vector; comparing the question query vector with a preset slice vector to obtain a target slice vector that matches the question query vector; and using the slice information corresponding to the target slice vector to generate text slices.

[0085] In this embodiment, in the process of using the question query information for text retrieval, the question query information can be vectorized to obtain a question query vector, wherein the vectorization can be semantic vector conversion, and the question query text in the question query information can be extracted and the semantic vector conversion can be performed on the question query text to obtain the question query vector; the question query vector can be compared with a preset slice vector to obtain a target slice vector matching the question query vector, wherein the slice vector represents a vector after the pre-configured text slice is vectorized, and the pre-configured text slice can be a pre-divided vehicle manual content into one or more text slices matching the vehicle function or problem. At this time, after comparing the question query vector with each preset slice vector, the similarity corresponding to each slice vector can be obtained. When the familiarity meets the preset similarity condition, the preset slice vector can be determined as the target slice vector, wherein the preset similarity condition can be a preset similarity threshold, or can be the top N similarities sorted from high to low, and of course can be other conditions, which are not specifically limited in this embodiment; and then the slice information corresponding to the target slice vector can be used to generate a text slice, and the text slice can contain text information matching the question query information in the vehicle manual content.

[0086] In one example, the question query information is vectorized to obtain the question query vector X. In a scenario where the preset slice vector includes slice vector A, slice vector B, slice vector C, and slice vector D, the question query vector is compared with the preset slice vector to obtain similarity A corresponding to slice vector A, similarity B corresponding to slice vector B, similarity C corresponding to slice vector C, and similarity D corresponding to slice vector D. At this time, if the preset similarity condition is the first two similarity conditions sorted from high to low, the similarities are sorted from high to low as follows: similarity B, similarity D, similarity A, and similarity C. At this time, it can be determined that similarity B and similarity D meet the preset similarity condition, and slice vector B and slice vector D can be determined as target slice vectors. Slice information B corresponding to slice vector B and slice information D corresponding to slice vector D can be used to generate text slices. If slice information B is an air-conditioning startup abnormality processing method and slice information D is an air-conditioning temperature abnormality processing method, the text slices include text content related to the air-conditioning startup abnormality processing method and text content related to the air-conditioning temperature abnormality processing method.

[0087] In an optional embodiment of the present application, intersection processing is performed based on the first image information and the second image information to obtain first alternative image information, which may specifically include the following sub-steps: obtaining at least one first image corresponding to the first image information, and obtaining at least one second image corresponding to the second image information; comparing each first image with each second image to obtain a first comparison result set, wherein the first comparison result set includes at least one first comparison result; when the first comparison result is an identical image result, determining the first image or the second image corresponding to the first comparison result as the first alternative image; and using the first alternative image to generate the first alternative image information.

[0088] In this embodiment, in the process of performing intersection processing based on the first image information and the second image information, at least one first image corresponding to the first image information and at least one second image corresponding to the second image information can be obtained, and each first image is compared with each second image to obtain a first comparison result set, wherein the first comparison result set includes at least one first comparison result, wherein the first comparison result is used to indicate whether the first image and the second image are the same, that is, the first comparison result can be an identical image result or a different image result, and in the method of comparing each first image with each second image, a pixel difference comparison can be used to obtain the number of difference pixels, and a first comparison result of the identical image result can be obtained when the number of difference pixels is less than a preset number threshold. A comparison result is obtained when the number of difference pixels is not less than a preset number threshold. Of course, other comparison methods can also be used, and this embodiment does not specifically limit this. Therefore, after obtaining the first comparison result, each first comparison result in the first comparison result set can be traversed to determine whether it is an image result. When the first comparison result is the same image result, it means that the first image and the second image compared at this time are the same image. Then, the first image or the second image corresponding to the first comparison result can be determined as the first candidate image. Since the first image and the second image can contain multiple identical images, one or more first candidate images can be obtained. Subsequently, a set of each first candidate image can be used to generate first candidate image information.

[0089] It should be noted that, in this embodiment, the first images and the second images are not necessarily related in quantity, and the comparison process is to compare each first image with each second image. For example, the first image includes a first image A, a first image B, and a first image C, and the second image includes a second image X and a second image Y. At this time, the comparison may be to compare the first image A with the second image X and the second image Y, respectively, and to compare the first image B with the second image X and the second image Y, respectively, and to compare the first image C with the second image X and the second image Y, respectively, thereby obtaining a first comparison result set consisting of six first comparison results, so that it can be determined whether the first image and the second image have the same image through each first comparison result.

[0090] In an optional embodiment of the present application, image retrieval is performed based on question query information to obtain first image information, which may specifically include the following sub-steps: vectorizing the question query information to obtain a question query vector; obtaining an image description vector; comparing the question query vector with the image description vector to obtain a target image description vector that matches the question query vector; and determining the image information corresponding to the target image description vector as the first image information.

[0091] In the present embodiment, during the process of image retrieval based on the question query information, the question query information may be vectorized to obtain a question query vector, wherein the vectorization may be semantic vector conversion, or may be extracting the question query text in the question query information and converting the question query text into a semantic vector to obtain the question query vector; and an image description vector is obtained to compare the question query vector with a preset image description vector to obtain a target image description vector matching the question query vector, wherein the image description vector represents a vector after the image description text corresponding to the vehicle description image is vectorized, and the vehicle description image may be an image contained in the vehicle manual; at this time, after comparing the question query vector with each preset image description vector, the similarity corresponding to each image description vector may be obtained, and when the familiarity meets the preset similarity condition, the preset image description vector may be determined as the target image description vector, wherein the preset similarity condition may be a preset similarity threshold, or may be the top N similarities sorted from high to low, and of course may be other conditions, which are not specifically limited in the present embodiment; and then the image information corresponding to the target image description vector may be determined as the first image information, so that the first image information may contain the image information matching the question query information in the vehicle manual content.

[0092] In one example, the question query information is vectorized to obtain the question query vector X. In a scenario where the preset image description vector includes image description vector A, image description vector B, image description vector C, and image description vector D, the question query vector is compared with the preset image description vector to obtain similarity A corresponding to image description vector A, similarity B corresponding to image description vector B, similarity C corresponding to image description vector C, and similarity D corresponding to image description vector D. At this time, if the preset similarity condition is the first two similarity conditions sorted from high to low, the similarities are sorted from high to low. The sequence is: similarity B, similarity D, similarity A, similarity C. At this time, it can be determined that similarity B and similarity D meet the preset similarity conditions, and the image description vector B and the image description vector D can be determined as the target image description vectors, and the image information B corresponding to the image description vector B and the image information D corresponding to the image description vector D can be used to generate text slices. If the image information B is an air-conditioning startup abnormality processing image, and the image information D is an air-conditioning temperature abnormality processing image, then the first image information includes image information related to the air-conditioning startup abnormality processing method and image information related to the air-conditioning temperature abnormality processing method.

[0093] In an optional embodiment of the present application, obtaining an image description vector may specifically include the following sub-steps: obtaining vehicle description information corresponding to the vehicle; extracting a vehicle description image from the vehicle description information, and obtaining an image description text corresponding to the vehicle description image; vectorizing the image description text to obtain an image description vector corresponding to the vehicle description image.

[0094] In this embodiment, in the process of pre-configuring the image description vector, the vehicle description information corresponding to the vehicle can be obtained. The vehicle description information can represent information including text content and image content of the vehicle manual. At this time, the vehicle description image can be extracted from the vehicle description information, and the image description text corresponding to the vehicle description image can be obtained, wherein the vehicle description image represents the image contained in the vehicle description information, and the image description text can represent the text used to describe the vehicle description image. The specific image description text can be text extracted from the vehicle description information to describe the vehicle description image, or it can be description text input separately according to each vehicle description image. Of course, the image description text corresponding to the vehicle description image can also be determined by other methods, and this embodiment does not make specific limitations on this; then the image description text can be vectorized to obtain the image description vector corresponding to the vehicle description image.

[0095] It should be noted that the image description vector in this embodiment can be pre-configured and generated according to the vehicle description information corresponding to the vehicle, or it can be generated in real time during the process of obtaining the first image information through image retrieval based on question query information. Of course, other methods can also be used, and this embodiment does not make any specific limitations on this.

[0096] In one example, in the present embodiment, when using question query information to perform image retrieval and text retrieval respectively, a preset database may be used for retrieval. The preset database may include a slice vector data table and an image description vector data table. The slice vector data table is a data table for storing slice vectors, and the image description vector data table is a data table for storing image description vectors. In the process of using question query information to perform image retrieval, the question query vector can be compared with each image description vector in the image description vector data table for retrieval, and in the process of using question query information to perform text retrieval, the question query vector can be compared with each slice vector in the slice vector data table for retrieval; thereby, one database can be used to complete image retrieval and text retrieval for question query information, which can effectively improve the efficiency of vehicle information query.

[0097] In an optional embodiment of the present application, intersection processing is performed based on the first alternative image information and the second alternative image information to obtain the target image information, which may specifically include the following sub-steps: obtaining at least one third image corresponding to the first alternative image information, and obtaining at least one fourth image corresponding to the second alternative image information; comparing each third image with each fourth image to obtain a second comparison result set, wherein the second comparison result set includes at least one second comparison result; when the second comparison result is an identical image result, determining the third image or the fourth image corresponding to the second comparison result as the second alternative image; and using the second alternative image to generate the target image information.

[0098] In this embodiment, in the process of performing intersection processing based on the third image information and the fourth image information, at least one third image corresponding to the third image information and at least one fourth image corresponding to the fourth image information can be obtained, and each third image is compared with each fourth image to obtain a second comparison result set, wherein the second comparison result set includes at least one second comparison result, wherein the second comparison result is used to indicate whether the third image and the fourth image are the same, that is, the second comparison result can be an identical image result or a different image result, and in a method of comparing each third image with each fourth image, a pixel difference comparison can be used to obtain the number of difference pixels, and a second comparison result of the identical image result can be obtained when the number of difference pixels is less than a preset number threshold. Second comparison result, when the number of difference pixels is not less than a preset number threshold, a second comparison result of a different image result is obtained. Of course, other comparison methods can also be used, and this embodiment does not specifically limit this; thus, after obtaining the second comparison result, each second comparison result in the second comparison result set can be traversed to determine whether it is an image result. When the second comparison result is the same image result, it means that the third image and the fourth image compared at this time are the same image, then the third image or the fourth image corresponding to the second comparison result can be determined as the second candidate image. Since the third image and the fourth image can contain multiple identical images, one or more second candidate images can be obtained; then the set of each second candidate image can be used to generate the second candidate image information.

[0099] It should be noted that the third images and the fourth images in this embodiment are not necessarily related in quantity, and the comparison process is to compare each third image with each fourth image. For example, the third image includes a third image A, a third image B, and a third image C, and the fourth image includes a fourth image X and a fourth image Y. At this time, the comparison may be to compare the third image A with the fourth image X and the fourth image Y, respectively, and to compare the third image B with the fourth image X and the fourth image Y, respectively, and to compare the third image C with the fourth image X and the fourth image Y, respectively, thereby obtaining a first comparison result set consisting of six first comparison results, so that it can be determined whether the third image and the fourth image have the same image through each first comparison result.

[0100] In an optional embodiment of the present application, before performing image retrieval based on the problem query information of the vehicle, the following sub-steps may be specifically included: obtaining vehicle description information corresponding to the vehicle; extracting vehicle description text from the vehicle description information; and segmenting the vehicle description text to obtain text slices.

[0101] In this embodiment, in the process of pre-configuring text slices, vehicle description information corresponding to the vehicle can be obtained. The vehicle description information can represent information containing text content and image content of the vehicle manual. At this time, the vehicle description text can be extracted from the vehicle description information, and the vehicle description text can be segmented according to a preset division strategy to obtain text slices, wherein the preset division strategy can be division according to vehicle functions, division according to problem types, division according to usage scenarios, etc., so that each text slice can contain the text corresponding to a specific content object. For example, when dividing according to vehicle functions, the text slices that can be obtained can be: air conditioning function description text, trunk control description text, wiper control description text, etc.

[0102] In an optional embodiment of the present application, step S120 performs semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information, which may specifically include the following sub-steps:

[0103] Generate prompt information based on question query information and text slices;

[0104] Inputting the prompt information into a preset semantic analysis model to obtain a model output result, wherein the model output result represents the text output by the semantic analysis model;

[0105] Based on the model output results, determine the text answer information.

[0106] In this embodiment, in the process of obtaining text answer information, prompt information can be generated based on the question query information and text slices, and the prompt information represents the prompt used to input into the semantic analysis model, so that the prompt information can be input into the preset semantic analysis model to obtain the model output result, wherein the semantic analysis model can be a large language processing model, such as a large language model (Large Language Models, LLM), and the model output result can represent the text output by the semantic analysis model; thereby, the text answer information can be determined based on the model output result, and specifically, the text corresponding to the model output result can be determined as the text answer information.

[0107] The vehicle information query method of the present application can be applied to a retrieval-augmented generation system (RAG) system, such as Figure 2 As shown, when applied to the RAG system, the specific implementation may include the following steps:

[0108] 1) Based on the questions raised by the driver and passengers (for example: What are the length, width and height of the vehicle?), the corresponding question query information is generated and vectorized to obtain a question query vector.

[0109] 2) Retrieval based on the questions raised by the driver and passengers includes two parts: 1) Retrieve images based on text slices: retrieve the topN text slices with the closest cosine distance between the question query vector and the text slice vector, where each slice contains an indefinite amount of first image information; 2) Direct retrieval based on the image description vector: retrieve the topN images with the closest cosine distance between the question query vector and the image description vector, that is, the second image information. Perform intersection processing on the first image information and the second image information to obtain the first candidate image information containing N first candidate images; where N is a positive integer greater than zero.

[0110] The topN text slices have been retrieved in the previous step. In this step, the specific slice content and question query information are input into the LLM model as prompt information, and the LLM autonomously generates text answer information related to the user's question. After obtaining the text answer information, the text answer information is vectorized to obtain a text answer vector, and image retrieval is performed based on the text answer vector, so as to retrieve the topN images with the closest cosine distance between the text answer vector and the image description vector, that is, the second candidate image information, and the second candidate image information is processed with the first candidate image information to obtain the target image information. Finally, the answer text information and the target image information can be used as the query results corresponding to the question query information, and output and displayed to the driver and passengers.

[0111] like Figure 3 As shown, the present application also discloses an embodiment, providing a vehicle information query device, including:

[0112] A first image retrieval module 310 is used to perform image retrieval based on the problem query information of the vehicle to obtain first candidate image information;

[0113] A semantic analysis module 320 is used to perform semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information;

[0114] A second image retrieval module 330, configured to perform image retrieval based on the answer text information to obtain second candidate image information;

[0115] An intersection processing module 340 is used to perform intersection processing based on the first candidate image information and the second candidate image information to obtain target image information;

[0116] The generation module 350 is used to generate a query result corresponding to the question query information based on the answer text information and the target image information.

[0117] In an optional embodiment of the present application, the first image retrieval module 310 may include:

[0118] A first image retrieval unit, configured to perform image retrieval based on the question query information to obtain first image information;

[0119] A text retrieval unit, used to perform text retrieval using the question query information to obtain the text slice;

[0120] A first determining unit, configured to determine preset image information corresponding to the text slice as second image information;

[0121] The first intersection processing unit is used to perform intersection processing based on the first image information and the second image information to obtain the first candidate image information.

[0122] In an optional embodiment of the present application, the text retrieval unit may include:

[0123] A first vectorization subunit is used to vectorize the question query information to obtain a question query vector;

[0124] A first comparison subunit, configured to compare the question query vector with a preset slice vector to obtain a target slice vector matching the question query vector;

[0125] The first generating subunit is used to generate the text slice using the slice information corresponding to the target slice vector.

[0126] In an optional embodiment of the present application, the first intersection processing unit may include:

[0127] A first acquisition subunit, configured to acquire at least one first image corresponding to the first image information, and acquire at least one second image corresponding to the second image information;

[0128] A second comparison subunit is used to compare each of the first images with each of the second images to obtain a first comparison result set, wherein the first comparison result set includes at least one first comparison result;

[0129] A first determining subunit, configured to determine the first image or the second image corresponding to the first comparison result as a first candidate image when the first comparison result is an identical image result;

[0130] The second generating subunit is configured to generate the first candidate image information by using the first candidate image.

[0131] In an optional embodiment of the present application, the first image retrieval unit may include:

[0132] A second vectorization subunit is used to vectorize the question query information to obtain a question query vector;

[0133] A second acquisition subunit is used to acquire an image description vector;

[0134] A third comparison subunit is used to compare the question query vector with the image description vector to obtain a target image description vector that matches the question query vector;

[0135] The second determining subunit is used to determine the image information corresponding to the target image description vector as the first image information.

[0136] In an optional embodiment of the present application, the second obtaining subunit may include:

[0137] A third acquisition subunit is used to acquire vehicle description information corresponding to the vehicle;

[0138] a fourth acquisition subunit, configured to extract a vehicle description image from the vehicle description information, and acquire an image description text corresponding to the vehicle description image;

[0139] The vectorization subunit is used to vectorize the image description text to obtain the image description vector corresponding to the vehicle description image.

[0140] In an optional embodiment of the present application, the intersection processing module 340 may include:

[0141] an acquiring unit, configured to acquire at least one third image corresponding to the first candidate image information, and acquire at least one fourth image corresponding to the second candidate image information;

[0142] a comparing unit, configured to compare each of the third images with each of the fourth images to obtain a second comparison result set, wherein the second comparison result set includes at least one second comparison result;

[0143] A second determining unit, configured to determine the third image or the fourth image corresponding to the second comparison result as a second candidate image when the second comparison result is an identical image result;

[0144] The first generating unit is configured to generate the target image information by using the second candidate image.

[0145] In an optional embodiment of the present application, the vehicle information query device may further include:

[0146] An acquisition module, used to acquire vehicle description information corresponding to the vehicle;

[0147] An extraction module, used for extracting vehicle description text from the vehicle description information;

[0148] The segmentation module is used to segment the vehicle description text to obtain the text slices.

[0149] In an optional embodiment of the present application, the semantic analysis module 320 may include:

[0150] A second generating unit, configured to generate prompt information according to the question query information and the text slice;

[0151] An input unit, used to input the prompt information into a preset semantic analysis model to obtain a model output result, wherein the model output result represents the text output by the semantic analysis model;

[0152] The third determination unit is used to determine the text answer information based on the model output result.

[0153] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0154] like Figure 4 As shown, the embodiment of the present application provides an air conditioner control device, including a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440.

[0155] Memory 430, for storing computer programs;

[0156] In one embodiment of the present application, the processor 410 is used to execute the program stored in the memory 430 to implement the vehicle information query control method provided by any of the aforementioned method embodiments, by performing image retrieval based on the question query information of the vehicle to obtain first candidate image information, and performing semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information, so that the question query information and the text slice can be semantically analyzed to obtain text answer information matching the question query information, without the need for the user to manually flip through the vehicle manual to find the relevant text; then, an image retrieval can be performed based on the answer text information to obtain second candidate image information, and an exchange can be performed based on the first candidate image information and the second candidate image information. The target image information is obtained by processing the set, and then the target image information matching the question query information can be obtained by processing the intersection of the first alternative image information and the second alternative image information, without the need for the user to manually look up the vehicle manual to find the relevant image; then, the query result corresponding to the question query information can be generated based on the answer text information and the target image information, thereby eliminating the need for the user to manually look up the vehicle manual, and adaptively generating a query result including the answer text represented by the answer text information and the target image represented by the target image information, which solves the problem of low efficiency of vehicle information query when manually looking up the vehicle manual, and can also play the role of jointly outputting images and texts, and can effectively improve the user's reading experience and the efficiency of vehicle information query.

[0157] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the vehicle information query method provided in any of the aforementioned method embodiments are implemented.

[0158] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution can be essentially or partly in other words, contribute to the relevant technology in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments, and obtain the first candidate image information by performing image retrieval based on the vehicle's question query information, and obtain text answer information by performing semantic analysis based on the question query information and the text slice corresponding to the question query information, so that the text answer information matching the question query information can be obtained by performing semantic analysis on the question query information and the text slice, without the user having to manually flip through the vehicle manual to find the relevant text; then the answer text information can be retrieved based on the answer text information. The image is retrieved based on the information to obtain the second alternative image information, and the intersection processing is performed based on the first alternative image information and the second alternative image information to obtain the target image information. Then, the target image information matching the question query information can be obtained by the intersection processing of the first alternative image information and the second alternative image information, without the need for the user to manually look up the vehicle manual to find the relevant image; then, the query result corresponding to the question query information can be generated based on the answer text information and the target image information, thereby eliminating the need for the user to manually look up the vehicle manual, and adaptively generating the query result including the answer text represented by the answer text information and the target image represented by the target image information, which solves the problem of low efficiency of vehicle information query when manually looking up the vehicle manual, and can also play the role of jointly outputting images and texts, thereby effectively improving the user's reading experience and the efficiency of vehicle information query.

[0160] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0161] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle information query method, characterized in that: include: Perform image retrieval based on the problem query information of the vehicle to obtain first candidate image information; Based on the question query information, semantic analysis is performed in combination with the text slice corresponding to the question query information to obtain text answer information; Performing image retrieval based on the answer text information to obtain second candidate image information; Performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information; Based on the answer text information and the target image information, a query result corresponding to the question query information is generated.

2. The vehicle information query method according to claim 1, characterized in that: The image retrieval is performed based on the problem query information of the vehicle to obtain the first candidate image information, including: Perform image retrieval based on the question query information to obtain first image information; Using the question query information to perform text retrieval to obtain the text slice; The text slice corresponds to preset image information, and is determined as second image information; The first candidate image information is obtained by performing intersection processing based on the first image information and the second image information.

3. The vehicle information query method according to claim 2, characterized in that: The step of using the question query information to perform text retrieval to obtain the text slice includes: Vectorizing the question query information to obtain a question query vector; Comparing the question query vector with a preset slice vector to obtain a target slice vector that matches the question query vector; The text slice is generated by using the slice information corresponding to the target slice vector.

4. The vehicle information query method according to claim 2, characterized in that: The performing intersection processing based on the first image information and the second image information to obtain the first candidate image information includes: Acquire at least one first image corresponding to the first image information, and acquire at least one second image corresponding to the second image information; Compare each of the first images with each of the second images to obtain a first comparison result set, wherein the first comparison result set includes at least one first comparison result; When the first comparison result is an identical image result, determining the first image or the second image corresponding to the first comparison result as a first candidate image; The first candidate image is used to generate the first candidate image information.

5. The vehicle information query method according to claim 2, characterized in that: Perform image retrieval based on the question query information to obtain first image information, including: Vectorizing the question query information to obtain a question query vector; Get the image description vector; Comparing the question query vector with the image description vector to obtain a target image description vector that matches the question query vector; The image information corresponding to the target image description vector is determined as the first image information.

6. The vehicle information query method according to claim 5, characterized in that: The step of obtaining an image description vector comprises: Obtaining vehicle description information corresponding to the vehicle; Extracting a vehicle description image from the vehicle description information, and obtaining an image description text corresponding to the vehicle description image; The image description text is vectorized to obtain the image description vector corresponding to the vehicle description image.

7. The vehicle information query method according to any one of claims 1 to 6, characterized in that: The performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information includes: Acquire at least one third image corresponding to the first candidate image information, and acquire at least one fourth image corresponding to the second candidate image information; Comparing each of the third images with each of the fourth images to obtain a second comparison result set, wherein the second comparison result set includes at least one second comparison result; When the second comparison result is an identical image result, determining the third image or the fourth image corresponding to the second comparison result as a second candidate image; The target image information is generated using the second candidate image.

8. The vehicle information query method according to any one of claims 1 to 6, characterized in that: Before performing image retrieval based on the problem query information of the vehicle, the method further includes: Obtaining vehicle description information corresponding to the vehicle; extracting a vehicle description text from the vehicle description information; The vehicle description text is sliced ​​to obtain the text slices.

9. The vehicle information query method according to any one of claims 1 to 6, characterized in that: The step of performing semantic analysis based on the question query information and combining the text slice corresponding to the question query information to obtain text answer information includes: Generate prompt information according to the question query information and the text slice; Inputting the prompt information into a preset semantic analysis model to obtain a model output result, wherein the model output result represents the text output by the semantic analysis model; Based on the model output result, the text answer information is determined.

10. A vehicle information query device, characterized in that: include: A first image retrieval module, used to perform image retrieval based on the problem query information of the vehicle to obtain first candidate image information; A semantic analysis module, used to perform semantic analysis based on the question query information and the text slice corresponding to the question query information to obtain text answer information; A second image retrieval module, used to perform image retrieval based on the answer text information to obtain second candidate image information; An intersection processing module, used for performing intersection processing based on the first candidate image information and the second candidate image information to obtain target image information; A generation module is used to generate a query result corresponding to the question query information based on the answer text information and the target image information.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the vehicle information query method described in any one of claims 1 to 9 when executing the program stored in the memory.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle information query method as described in any one of claims 1 to 9 is implemented.

Citation Information

Cited By

  • Image searching method and device, intelligent agent, electronic equipment and storage medium

    CN119597955A