Hotel room search method, system, device, medium and product
By combining the room type pictures and structured information of hotel rooms, and using multimodal large language model for comparison and matching learning, the problem of low search efficiency and low accuracy of similar room types when booking hotel rooms is solved, achieving more efficient and accurate search results.
Patent Information
- Application Number
- CN202411975209.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-04
AI Technical Summary
The search for similar room types when booking existing hotel rooms is inefficient and low accuracy, making it difficult for users to find alternative or same room types that meet their expectations.
By collecting room type pictures and structured information of hotel rooms, establishing a mapping relationship between pictures and structured information, using multimodal large language model for comparison and matching learning, and building a hotel room retrieval model to improve the search efficiency and accuracy of similar room types.
It realizes more accurately capturing and matching user query intentions, improves the search efficiency and accuracy of hotel rooms similar to the room type to be queried, and improves the relevance of search results.
Smart Images
Figure CN119760163A_ABST
Abstract
Description
Background Art
[0002] In the hotel industry, online travel platforms (OTA) have become the main way for modern consumers to book hotel rooms. When consumers choose a room type, in addition to price factors, they will also make decisions based on the specific characteristics of the room type, such as room size, bed type, room facilities, etc.
[0003] When browsing rooms, customers often find that their favorite rooms are overpriced or sold out. At this time, they need to find other comparable or similar rooms that meet their expectations. However, current users lack an efficient path to find similar rooms. If you search based solely on room images, you will not be able to use the room-related structured information, such as room name, number of beds, room area, etc., which is crucial for recalling truly similar rooms. Traditional single-modal retrieval methods rely on keyword searches or image features, which are inefficient and have limited accuracy.
[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0005] In view of this, the present invention provides a hotel room search method, system, device, medium and product to at least solve the problem of low efficiency and low accuracy in the existing hotel room reservation search for similar room types.
[0006] In one aspect, an embodiment of the present invention provides a hotel room search method, comprising:
[0007] Collecting room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each room type picture, and establishing a mapping relationship between the room type pictures and the room type structured information;
[0008] Constructing room type image sample pairs based on the image feature extraction model, and constructing multimodal sample pairs based on room type structured information corresponding to each room type image in the room type image sample pairs;
[0009] The contrastive learning and matching learning strategies are used to optimize the parameters of the multimodal large language model, and a hotel room retrieval model is trained based on multimodal sample pairs.
[0010] The room type image and room type structured information to be queried are input into the hotel room retrieval model to obtain the corresponding hotel room from the hotel rooms for sale.
[0011] In some embodiments, the room type image includes: bedroom, living room, bathroom or kitchen;
[0012] The structured information of a room type includes: room name, room area, number of occupants, bed type name, number of beds, bed width, smoking policy or window type.
[0013] In some embodiments, constructing a room type image sample pair based on an image feature extraction model includes:
[0014] Extract image features from room images based on a similarity model, and store the extracted image features in a vector database;
[0015] The similarity between the vectors of each image feature is calculated based on a vector search engine, and based on the similarity between the image feature vectors of at least two room type images being greater than a preset value, a combination of at least two room type images is used as a room type image sample pair.
[0016] In some embodiments, the similarity between the feature vectors of each picture is calculated based on a vector search engine, and based on the similarity between the picture feature vectors of at least two room type pictures being greater than a preset value, a combination of at least two room type pictures is used as a room type picture sample pair, and then, the method further includes:
[0017] Based on the image recognition model, hotel facility features are extracted from the room images in each room image sample pair, and room image sample pairs whose similarity of hotel facility features is less than a preset value are deleted.
[0018] In some embodiments, the multimodal large language model includes:
[0019] An image encoder, used to extract room image features from room images;
[0020] A large language model is used to analyze the room type image features and the room type structured information corresponding to the room type image, and to extract the room type multimodal features;
[0021] Feature mapping module, used to aggregate multimodal features of room types.
[0022] In some embodiments, contrastive learning and matching learning strategies are used to optimize the parameters of the multimodal large language model, and a hotel room retrieval model is trained based on multimodal sample pairs, including:
[0023] Taking the hidden layer features as multimodal features, adding a sample comparison learning head and a sample matching learning head to the multimodal large language model, and fine-tuning the multimodal large language model, wherein the target parameters of fine-tuning include the fully connected layer and the attention matrix in the attention module;
[0024] Design targeted optimization targets for the multimodal large language model, including contrastive learning loss and sample matching loss. Contrastive learning loss is used to shorten the distance between positive sample pairs and increase the distance between negative sample pairs. A positive sample pair refers to two samples that match the query sample and the target sample, and a negative sample pair refers to two samples that do not match the query sample and the target sample. Sample matching loss is used to train the multimodal large language model to recognize whether two samples come from the same sample pair.
[0025] Optimize the parameters of the multimodal large language model using multimodal sample pairs in instruction format collected in an unsupervised manner;
[0026] The multimodal large language model with optimized parameters is used as the hotel room retrieval model.
[0027] On the other hand, an embodiment of the present invention further provides a hotel room retrieval system, comprising:
[0028] A data collection module collects room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each room type picture, and establishes a mapping relationship between the room type pictures and the room type structured information;
[0029] A data processing module constructs room type image sample pairs based on the image feature extraction model, and constructs multimodal sample pairs based on room type structured information corresponding to each room type image in the room type image sample pairs;
[0030] The model training module uses contrastive learning and matching learning strategies to optimize the parameters of the multimodal large language model and trains a hotel room retrieval model based on multimodal sample pairs.
[0031] The room retrieval module inputs the room type image and room type structured information to be queried into the hotel room retrieval model, and obtains the corresponding hotel room from the hotel rooms for sale.
[0032] In another aspect, an embodiment of the present invention further provides a hotel room search device, comprising:
[0033] processor;
[0034] a memory storing computer-readable instructions;
[0035] The processor is configured to perform the above method by executing computer-readable instructions.
[0036] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium storing computer-readable instructions, which implement the above method when executed by a processor.
[0037] On the other hand, an embodiment of the present invention further provides a computer program product, including computer-readable instructions, which implement the above method when executed by a processor.
[0038] The hotel room retrieval method, system, device, medium and product of the present invention, by combining hotel room type pictures and room type structured information, adopts a multimodal retrieval large model, can more accurately capture and match the user's query intention, improve the efficiency and accuracy of retrieving hotel rooms similar to the room type pictures to be queried, and improve the relevance of the retrieval results provided to the user. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0040] Figure 1 It is a flowchart of the steps of a hotel room search method provided by the present invention;
[0041] Figure 2 yes Figure 1 Specific step flow chart of step S120;
[0042] Figure 3 yes Figure 1 Specific step flow chart of step S130;
[0043] Figure 4 yes Figure 1 A schematic diagram of the specific steps of step S130;
[0044] Figure 5 It is a module schematic diagram of a hotel room retrieval system provided by the present invention;
[0045] Figure 6 It is a structural schematic diagram of a hotel room retrieval device according to an embodiment of the present invention;
[0046] Figure 7 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION
[0047] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.
[0048] The words "first", "second" and the like used in the specific description do not indicate any order, quantity or importance, but are only used to distinguish different components. In addition, in the description of the present invention, the orientation or position relationship indicated by the terms "upper" and "lower" are based on the orientation or position relationship shown in the drawings, which are only for the convenience of description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation of the present invention.
[0049] It should be noted that, in the absence of conflict, the embodiments of the present invention and features in different embodiments may be combined with each other.
[0050] On one hand, Figure 1 As shown, an embodiment of the present invention provides a hotel room search method, comprising:
[0051] S110, collecting room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each room type picture, and establishing a mapping relationship between the room type pictures and the room type structured information;
[0052] S120, constructing room type picture sample pairs based on the picture feature extraction model, and constructing multimodal sample pairs based on room type structured information corresponding to each room type picture in the room type picture sample pairs;
[0053] S130, using contrastive learning and matching learning strategies to optimize the parameters of the multimodal large language model, and training a hotel room retrieval model based on multimodal sample pairs;
[0054] S140: Input the room type image and room type structured information to be queried into a hotel room retrieval model, and obtain the corresponding hotel room from the hotel rooms for sale.
[0055] It should be noted that the Multimodal Large Language Model (MLLM) is a model extended from the Large Language Model (LLM) that has the ability to receive and reason about multimodal information. Compared with the unimodal LLM, the MLLM has the following advantages: it is more in line with the human habit of understanding the world; it has a more powerful and user-friendly interface; and it supports a wider range of tasks.
[0056] The above settings, by combining hotel room pictures and room type structured information, use a multimodal retrieval large model to more accurately capture and match the user's query intent, improve the efficiency and accuracy of retrieving hotel rooms similar to the room type pictures to be queried, and improve the relevance of the search results provided to users. In addition, the above settings can also provide users with more personalized and efficient hotel room retrieval services, enrich hotel booking methods, allow users to experience a more flexible and convenient booking process, and enhance the user's booking experience.
[0057] It is worth noting that the above S110 to S140 are merely step numbers, which are used to facilitate reference and avoid text duplication. Unless otherwise specified, the above and subsequent step numbers will not limit the order of implementation of the various steps of the method. In other embodiments, the above steps of the method can also be written and implemented in an interchangeable order, and are not limited thereto.
[0058] In some embodiments, the room type structured information includes: room type pictures including: bedroom, living room, bathroom or kitchen. Room type name, room type area, number of occupants, bed type name, number of beds, bed width, smoking regulations or window type. Specifically, room type pictures and room type structured information can be obtained from the database of the OTA platform. The obtained room type pictures and room type structured information cover hotel rooms of different regions, different star ratings and different styles to ensure that the hotel room retrieval model can be generalized to various practical application scenarios.
[0059] like Figure 2 As shown, in some embodiments, the "constructing room type image sample pairs based on the image feature extraction model" in step 120 includes:
[0060] S121, extracting picture features from the room type picture based on the similarity model, and storing the extracted picture features in a vector database;
[0061] S122. Calculate the similarity between the vectors of each image feature based on a vector search engine, and based on the similarity between the vectors of the image features of at least two room images being greater than a preset value, use a combination of the at least two room images as a room image sample pair.
[0062] For the above step S121, the similarity model may be CLIP (Contrastive Language-Image Pretraining) or DINO (DETR with Improved deNoising anchOr boxes), etc., and the present application does not impose any limitation on this.
[0063] The purpose of the above step S122 is to select room type picture pairs with high similarity in each model as candidate room type picture sample pairs.
[0064] Continue to refer Figure 2 In some embodiments, after the above step S122, the method further includes:
[0065] S123. Extract hotel facility features from the room images in each room image sample pair based on the image recognition model, and delete room image sample pairs whose similarity of hotel facility features is less than a preset value.
[0066] Specifically, the image recognition model can be an image scene classification model or an image facility recognition model, etc., which is used to recognize scenes or facilities such as bedrooms, living rooms, toilets or kitchens in room images. The above settings are used to perform secondary filtering on candidate room image sample pairs to obtain final room image sample pairs, avoiding problematic data with high similarity in room images but large differences in facilities (e.g., inconsistent number of beds), and ensuring consistency of image content in room image sample pairs.
[0067] For the above steps S110 to S130, it should be noted that this series of data acquisition and processing steps of room type pictures and room type structured information are all completed in an unsupervised manner, which minimizes manual intervention and ensures the objectivity and comprehensiveness of data acquisition. At the same time, the unsupervised approach can also reduce the dependence on a large amount of labeled data, reduce training costs, and improve the customization capabilities of multimodal large language models.
[0068] Furthermore, the data set of room type image sample pairs is embodied in the form of instructions, and includes the storage path of the room type images and the room type structured information. For example: "Room type information: {'image': , 'Room Name': 'View Four-Bedroom Suite', 'Room Area': 360, 'Stars': 4, 'Number of Guests': 8, 'Bed Type': 'Double Bed', 'Number of Beds': 4, 'Bed Width': 2.0, 'Smoking Policy': 'Smoking', 'Window': 'With Window'}, summarize this room type in one word: <emb>”.
[0069] In some embodiments, the multimodal large language model includes: an image encoder, a large language model, and a feature mapping module. The image encoder is used to extract room type picture features of room type pictures. The large language model is used to parse room type picture features and room type structured information corresponding to the room type pictures, and extract room type multimodal features. The feature mapping module is used to aggregate room type multimodal features.
[0070] like Figure 3 and Figure 4 As shown, in some embodiments, step S130 includes:
[0071] S131, using the hidden layer features as multimodal features, adding a sample comparison learning head and a sample matching learning head to the multimodal large language model, and fine-tuning the multimodal large language model, wherein the fine-tuning target parameters include a fully connected layer and an attention matrix in an attention module;
[0072] S132. Design targeted optimization targets for the multimodal large language model, the optimization targets including: contrastive learning loss and sample matching loss; wherein, contrastive learning loss is used to shorten the distance between positive sample pairs and lengthen the distance between negative sample pairs, a positive sample pair refers to two samples that match a query sample and a target sample, a negative sample pair refers to two samples that do not match a query sample and a target sample, and sample matching loss is used to train the multimodal large language model to recognize whether two samples are from the same sample pair;
[0073] S133, optimizing parameters of a multimodal large language model using multimodal sample pairs in instruction format collected in an unsupervised manner;
[0074] S134. Using the multimodal large language model with optimized parameters as a hotel room retrieval model.
[0075] For the above step S131, specifically, due to the autoregressive nature of the multimodal large language model, the present invention adopts <emb>The last hidden layer feature of the token is used as the global multimodal feature of the room type, and the multimodal features are mapped to the contrast feature space and the matching feature space respectively by adding two modules, the sample contrast learning head and the sample matching learning head, on the basis of the original multimodal large language model structure. In addition, the present invention uses LoRA (Low-Rank Adaptation) technology for a multimodal large language model to improve the room type feature extraction effect while increasing the batch size during training. The fine-tuning target parameters include the fully connected layer and the attention matrix in the attention module. The attention module is the module in the multimodal large language model that is responsible for processing and understanding room type images and room type structured information.
[0076] The above step S132 is intended to enhance the multimodal large language model's ability to understand and integrate data of different modalities. Specifically, the calculation formula of contrastive learning loss is as follows:
[0077]
[0078] in, is the learning loss value, B is the training batch size, τ represents the learnable temperature parameter, n i and n j are the i-th and j-th samples, For n i The other sample that constitutes the positive pair.
[0079] In practical applications, randomly selected negative samples may be too simple, resulting in poor learning effect of multimodal large language models. In order to solve this problem, the present invention adopts a difficult sample mining strategy, that is, based on the feature similarity sim(a, b) between samples, a multinomial distribution sampling method is used to select negative samples, so that the more similar the negative samples are to the positive samples, the more likely they are to be selected, thereby increasing the difficulty of training the multimodal large language model. The calculation formula of sample matching loss is as follows:
[0080]
[0081] in, is the matching loss value, B is the training batch size, y i is the true value, is the predicted value.
[0082] For the above step S133, specifically, the goal of fine-tuning the multimodal large language model is to transform the conversational capabilities of the pre-trained model into the ability to extract features of multi-granular room images and room structured information. During the model fine-tuning process, the multimodal sample pair dataset in the instruction format collected in an unsupervised manner is used to perform end-to-end parameter optimization on the multimodal large language model.
[0083] For the above step S134, specifically, after the fine-tuning training stage, the multimodal large language model can accurately capture and associate room information of different modalities, that is, room pictures and room structured information, and finally obtain a hotel room retrieval model. By inputting the room pictures and structured information corresponding to the hotel room, the hotel room retrieval model can fully complete the extraction of features of different granularities and finally output a multimodal feature vector.
[0084] The hotel room retrieval method of the present invention, by combining hotel room type pictures and room type structured information, adopts a multimodal retrieval large model, can more accurately capture and match the user's query intention, improve the efficiency and accuracy of retrieving hotel rooms similar to the room type pictures to be queried, and improve the relevance of the retrieval results provided to the user.
[0085] like Figure 5 As shown, on the other hand, an embodiment of the present invention further provides a hotel room retrieval system 50, comprising:
[0086] The data collection module 51 collects room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each room type picture, and establishes a mapping relationship between the room type pictures and the room type structured information;
[0087] The data processing module 52 constructs room type image sample pairs based on the image feature extraction model, and constructs multimodal sample pairs based on room type structured information corresponding to each room type image in the room type image sample pairs;
[0088] The model training module 53 uses contrastive learning and matching learning strategies to optimize the parameters of the multimodal large language model, and trains a hotel room retrieval model based on multimodal sample pairs;
[0089] The room search module 54 inputs the room type image and room type structured information to be queried into the hotel room search model, and obtains the corresponding hotel room from the hotel rooms for sale.
[0090] The technical effects of the hotel room search system of the present invention can be referred to the aforementioned hotel room search method embodiment, which will not be described in detail here.
[0091] like Figure 6 As shown, in another aspect, an embodiment of the present invention further provides a hotel room search device, comprising: a processor; a memory storing computer-readable instructions, wherein the processor is configured to execute a hotel room search method by executing the computer-readable instructions.
[0092] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".
[0093] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0094] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.
[0095] The storage unit 620 stores computer-readable instructions, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps of various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 The steps of the method shown in .
[0096] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit 6201 and / or a cache storage unit 6202 , and may further include a read-only storage unit (ROM) 6203 .
[0097] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0098] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0099] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.
[0100] The technical effects of the hotel room search device of the present invention can be referred to the aforementioned hotel room search method embodiment, which will not be described in detail here.
[0101] like Figure 7 As shown, in another aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-readable instructions. A processor of a computing device can read the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computing device executes the hotel room retrieval method described in the above embodiments.
[0102] The computer-readable storage medium of the present invention can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0103] A computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable instructions. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable instructions contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
[0104] The computer-readable storage medium of the embodiment of the present invention includes computer-readable instructions, which can be written in any combination of one or more programming languages. Programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0105] The technical effects of the computer-readable storage medium of the present invention can be referred to the aforementioned hotel room search method embodiment, which will not be described in detail here.
[0106] In another aspect, an embodiment of the present invention further provides a computer program product, which includes computer-readable instructions stored in a computer-readable storage medium. A processor of a computing device can read the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computing device executes the hotel room retrieval method described in the above embodiments.
[0107] The specific programming language written form of the computer-readable instructions included in the computer program product of the embodiment of the present invention can refer to the computer-readable instructions included in the aforementioned computer-readable storage medium embodiment, and will not be repeated here.
[0108] The technical effects of the computer program product of the present invention can be referred to the aforementioned hotel room search method embodiment, which will not be described in detail here.
[0109] In summary, the hotel room retrieval method, system, device, medium and product of the present invention, by combining the room type pictures and room type structured information of the hotel room, adopts a multimodal retrieval large model, which can more accurately capture and match the user's query intention, improve the efficiency and accuracy of retrieving hotel rooms similar to the room type pictures to be queried, and improve the relevance of the retrieval results provided to the user.
[0110] The above contents are further detailed descriptions of the present invention in combination with specific optional implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.< / emb> < / emb>
Claims
1. A hotel room search method, characterized in that: include: Collecting room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each of the room type pictures, and establishing a mapping relationship between the room type pictures and the room type structured information; Constructing room type picture sample pairs based on the picture feature extraction model, and constructing multimodal sample pairs based on the room type structured information corresponding to each room type picture in the room type picture sample pairs; The parameters of the multimodal large language model are optimized by using contrastive learning and matching learning strategies, and a hotel room retrieval model is trained based on the multimodal sample pairs; The room type picture and room type structured information to be queried are input into the hotel room retrieval model, and the corresponding hotel room is obtained from the hotel rooms for sale.
2. The hotel room search method according to claim 1, characterized in that: The room pictures include: bedroom, living room, bathroom or kitchen; The room type structured information includes: room type name, room type area, number of occupants, bed type name, number of beds, bed width, smoking regulations or window type.
3. The hotel room search method according to claim 1, characterized in that: The constructing room type picture sample pairs based on the picture feature extraction model includes: Extracting picture features from the room type picture based on a similarity model, and storing the extracted picture features in a vector database; The similarity between the vectors of each of the image features is calculated based on a vector search engine, and based on the similarity between the vectors of the image features of at least two of the room pictures being greater than a preset value, a combination of at least two of the room pictures is used as a sample pair of the room pictures.
4. The hotel room search method according to claim 3, characterized in that: The vector search engine is used to calculate the similarity between the feature vectors of each picture, and based on the similarity between the feature vectors of at least two room pictures being greater than a preset value, a combination of at least two room pictures is used as the room picture sample pair, and then the method further includes: Based on the picture recognition model, hotel facility features are extracted from the room pictures in each room picture sample pair, and the room picture sample pairs whose similarity of hotel facility features is less than a preset value are deleted.
5. The hotel room search method according to claim 1, characterized in that: The multimodal large language model includes: An image encoder, used for extracting room type picture features of the room type picture; A large language model is used to parse the room type picture features and the room type structured information corresponding to the room type picture, and extract room type multimodal features; A feature mapping module is used to aggregate the multimodal features of the room type.
6. The hotel room search method according to claim 5, characterized in that: The contrastive learning and matching learning strategies are used to optimize the parameters of the multimodal large language model, and based on the multimodal sample pairs, a hotel room retrieval model is trained, including: Using hidden layer features as the multimodal features, adding a sample comparison learning head and a sample matching learning head to the multimodal large language model, and fine-tuning the multimodal large language model, wherein the target parameters of the fine-tuning include a fully connected layer and an attention matrix in an attention module; Designing targeted optimization targets for the multimodal large language model, the optimization targets include: contrastive learning loss and sample matching loss; wherein the contrastive learning loss is used to shorten the distance between positive sample pairs and increase the distance between negative sample pairs, the positive sample pair refers to two samples that match the query sample and the target sample, the negative sample pair refers to two samples that do not match the query sample and the target sample, and the sample matching loss is used to train the multimodal large language model to recognize whether the two samples are from the same sample pair; Optimizing parameters of the multimodal large language model using the multimodal sample pairs in the instruction format collected in an unsupervised manner; The multimodal large language model with optimized parameters is used as the hotel room retrieval model.
7. A hotel room search system, characterized in that: include: A data collection module collects room type pictures of multiple hotel rooms and room type structured information of the hotel rooms corresponding to each of the room type pictures, and establishes a mapping relationship between the room type pictures and the room type structured information; A data processing module, constructing room type picture sample pairs based on the picture feature extraction model, and constructing multimodal sample pairs based on the room type structured information corresponding to each room type picture in the room type picture sample pairs; The model training module uses contrastive learning and matching learning strategies to optimize the parameters of the multimodal large language model, and trains a hotel room retrieval model based on the multimodal sample pairs; The room search module inputs the room type image and room type structured information to be queried into the hotel room search model, and obtains the corresponding hotel room from the hotel rooms for sale.
8. A hotel room search device, characterized in that: include: processor; a memory storing computer-readable instructions; The processor is configured to perform the method according to any one of claims 1 to 6 by executing the computer readable instructions.
9. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product comprising computer readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.