Method, device and machine readable storage medium for recognizing points of interest in a photograph
By combining an interest point classification model and a signboard segmentation model, interest points in photos are automatically identified, solving the problems of low efficiency and poor accuracy of manual judgment and achieving efficient and low-cost interest point recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAVINFO
- Filing Date
- 2023-08-03
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, manually identifying points of interest in photos is inefficient and prone to errors, resulting in high labor costs and failing to meet practical needs.
A point-of-interest (POI) classification model is used to filter photos, and a sign segmentation model is used to mark the outline of the region where the POI is located. By combining model training and parameter transfer, the efficiency and accuracy of automated recognition are improved.
It significantly reduces labor costs, improves the processing efficiency and accuracy of interest point identification in photos, and meets the high-efficiency requirements of map updates.
Smart Images
Figure CN117290538B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of interest point recognition technology in photographs, and more particularly to a method, apparatus, and machine-readable storage medium for recognizing interest points in photographs. Background Technology
[0002] Points of Interest (POIs) in a map can change. To maintain the timeliness of POIs, a large amount of imagery needs to be collected, POIs extracted from the images, and then compared with their corresponding counterparts on the map. If they don't match, the extracted POIs are used to replace the corresponding POIs on the map. In the process of extracting POIs from images, existing methods involve manually determining whether POIs exist in the images. For images containing POIs, the contours of the regions where POIs are located need to be manually marked to extract the POIs from the marked regions. However, manually determining the presence of POIs and marking their contours is inefficient, requires significant manpower, and is prone to errors in judging the presence of POIs and marking their contours when staff are fatigued, failing to meet practical needs. Summary of the Invention
[0003] In view of this, embodiments of this specification provide a method, apparatus, and machine-readable storage medium for identifying points of interest in photographs, in order to solve the problems of high cost and error-proneness in marking points of interest and marking the outlines of points of interest in photographs in the prior art.
[0004] The embodiments in this specification adopt the following technical solutions:
[0005] This specification provides an embodiment of a method for identifying points of interest in a photograph, including:
[0006] Obtain the original photos used to update points of interest on the map;
[0007] The original photos are filtered using an interest point classification model to obtain target photos containing interest points; wherein, the interest point classification model is used to determine whether there are interest points in the original photos.
[0008] The outline of the region where the point of interest is located on the target photo is marked by the sign segmentation model to obtain the segmented region of the point of interest on the target photo; wherein, the sign segmentation model is used to segment the region where the point of interest is located on the target photo.
[0009] Identify points of interest within the segmented region.
[0010] Optionally, before filtering the original photos using the point of interest classification model, the method further includes:
[0011] Obtain the first set of photos containing annotation information for points of interest;
[0012] The interest point classification model is trained using the first set of photos.
[0013] Optionally, before marking the contour of the region containing the point of interest on the target photograph using the sign segmentation model, the method further includes:
[0014] Obtain a second set of photos containing annotation information for the regions where points of interest are located;
[0015] The sign segmentation model is trained using the second set of photos.
[0016] Optionally, the photos in the second photo set are at least a portion of the photos in the first photo set.
[0017] Optionally, before training the sign segmentation model using the second set of photos, the method further includes:
[0018] The model parameters of the trained interest point classification model are transferred to the sign segmentation model to obtain the sign segmentation model with transferred parameters; wherein, the model parameters are used to determine whether there are interest points in the photo;
[0019] The step of training the sign segmentation model using the second set of photos includes:
[0020] The sign segmentation model with the transferred parameters is trained using the second set of photos.
[0021] Optionally, the filtering of the original photos using the point of interest classification model includes:
[0022] The identification of photos containing points of interest in the original photos is recorded using a point of interest classification model, and photos whose identification is not recorded are removed.
[0023] Optionally, the filtering of the original photos using the point of interest classification model includes:
[0024] Intermediate features of the original photo are extracted using an interest point classification model, wherein the intermediate features are image pixels of the original photo represented in vector form;
[0025] The original photo is scored based on the intermediate features of the original photo using the point of interest classification model. Photos with scores below a preset threshold are removed, as they do not contain points of interest.
[0026] Optionally, marking the contour of the region containing the point of interest in the target photo using the sign segmentation model includes:
[0027] The signboard segmentation model identifies the location of points of interest in the target photo based on the intermediate features of the target photo.
[0028] The outline of the region containing the point of interest in the target photograph is marked.
[0029] This specification also provides an apparatus for identifying points of interest in a photograph, comprising:
[0030] The photo acquisition module is used to acquire the original photos for updating points of interest on the map.
[0031] The photo filtering module is used to filter the original photos using an interest point classification model to obtain target photos containing interest points; wherein, the interest point classification model is used to determine whether there are interest points in the original photos.
[0032] The point of interest segmentation module is used to mark the outline of the region where the point of interest is located on the target photo using a sign segmentation model, thereby obtaining the segmented region of the point of interest on the target photo; wherein, the sign segmentation model is used to segment the region where the point of interest is located on the target photo;
[0033] The point of interest (POI) identification module is used to identify POIs in the segmented region.
[0034] This specification also provides a machine-readable storage medium storing a machine-executable program and / or instructions that, when executed by a processor, implement the method for identifying points of interest in a photograph according to any of the preceding claims.
[0035] The above-described at least one technical solution used in the embodiments of this specification can achieve the following beneficial effects:
[0036] The system acquires original photos taken by street view image acquisition devices to update points of interest (POIs) on the map. It then filters the original photos using a POI classification model to obtain target photos containing POIs. Finally, it marks the outlines of the regions containing POIs in the target photos using a sign segmentation model to obtain segmented regions of POIs in the target photos. The system identifies POIs within these segmented regions, achieving very high processing efficiency and accuracy while significantly reducing manual labor costs. Attached Figure Description
[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0038] Figure 1 This is a flowchart illustrating a method for identifying points of interest in a photograph according to one embodiment of this specification.
[0039] Figure 2 This is a schematic diagram of the architecture of a method for identifying points of interest in a photograph according to one embodiment of this specification;
[0040] Figure 3 This is a schematic block diagram of the device for identifying points of interest in a photograph according to one embodiment of this specification;
[0041] Figure 4 This is a comparative diagram showing an embodiment of the present specification that does not include an interest point classification model and includes an interest point classification model.
[0042] Figure 5 This diagram illustrates a comparison between the training cost of the combined training cost of the sign segmentation model and the interest point classification model in one embodiment of this specification and the training cost of training the sign segmentation model alone.
[0043] Figure 6 This is a schematic diagram of the human-machine interface for storing target photos in a Redis database according to one embodiment of this specification. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0045] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0046] Figure 1 This is a flowchart illustrating a method for identifying points of interest in a photograph according to one embodiment of this specification. See also... Figure 1 Methods for identifying points of interest in a photograph may include:
[0047] Step S102: Obtain the original photos used to update points of interest on the map;
[0048] In this step, the original photos can be captured using image acquisition equipment. This equipment can be a professional data acquisition vehicle with a camera, a vehicle-mounted dashcam, or a camera used by crowdsourcing users to take photos. Points of Interest (POIs), commonly known as locations, can consist of coordinates and names, such as houses, shops, mailboxes, or bus stops on a map; POIs can also be understood as signs. Generally, the acquired original photos can be stored locally. The original photos can be in JPG or PNG format.
[0049] Step S104: Filter the original photos using the point of interest classification model to obtain target photos containing points of interest; wherein, the point of interest classification model is used to determine whether there are points of interest in the original photos.
[0050] In this step, the original photos are filtered using an interest point classification model to obtain target photos containing interest points. This avoids original photos that do not contain interest points from entering the sign segmentation model, reducing the processing of unnecessary photos and improving the overall image processing efficiency. For example, see... Figure 4 , Figure 4 This is a comparative diagram showing an embodiment of the present specification that does not include an interest point classification model and includes an interest point classification model. Figure 4 Scheme B in this embodiment is as follows: the point of interest classification model excludes photos that do not contain points of interest, ensuring that photos that contain points of interest are included in the sign segmentation model, and the sign segmentation model marks the outline of the area where the points of interest are located. Figure 4 Option A represents other solutions. All original photos are entered into the sign segmentation model, which requires marking the outline of the area where the point of interest is located in all photos, resulting in low efficiency.
[0051] Step S106: Mark the outline of the region where the point of interest is located on the target photo using the sign segmentation model to obtain the segmented region of the point of interest on the target photo; wherein, the sign segmentation model is used to segment the region where the point of interest is located on the target photo.
[0052] In this step, the outline of the region where the point of interest is located on the target photo is marked by the sign segmentation model, so as to obtain the segmented region of the point of interest on the target photo, so as to identify the point of interest in the segmented region and improve the identification efficiency of the point of interest.
[0053] Step S108: Identify points of interest in the segmented region.
[0054] In this embodiment, original photos taken by a street view image acquisition device are used to update points of interest on the map. The original photos are then filtered using a point of interest classification model to obtain target photos containing points of interest. The outlines of the areas containing points of interest in the target photos are marked using a sign segmentation model to obtain segmented regions of points of interest in the target photos. The points of interest in the segmented regions are then identified, which has very high processing efficiency and accuracy, and significantly reduces labor costs.
[0055] In the embodiments of this specification, before filtering the original photos using the point of interest classification model, the following may also be included:
[0056] Obtain the first set of photos containing annotation information for points of interest;
[0057] The interest point classification model was trained using the first set of photos.
[0058] In this embodiment, the annotation information for points of interest can be understood as marking the presence of points of interest on the first image. Generally, the first image set is obtained by annotating the points of interest in the original images. The first image set is used to train the point of interest classification model, enabling the model to accurately determine whether the original image contains points of interest, eliminating the need for manual annotation of whether images contain points of interest, thus saving labor costs and improving efficiency.
[0059] In the embodiments of this specification, before marking the contour of the region where the point of interest is located on the target photo using the sign segmentation model, the following may also be included:
[0060] Obtain a second set of photos containing annotation information for the regions where points of interest are located;
[0061] The signboard segmentation model was trained using a second set of photos.
[0062] In this embodiment, the regions containing points of interest (POIs) in the original photographs are typically labeled to obtain a second photograph set. The labeling information for the POI regions can be understood as marking the outlines of these regions in the second photograph, or as selecting the outlines of the POI regions in the second photograph. The sign segmentation model is trained using the second photograph set, enabling it to label the outlines of the POI regions without requiring manual labeling, thus saving labor costs and improving efficiency.
[0063] In the embodiments described in this specification, the photos in the second photo set may be at least a portion of the photos in the first photo set.
[0064] In this embodiment, the photos in the second photo set are a portion of the photos in the first photo set, which can reduce the requirement for the number of original photos to be acquired and further improve processing efficiency.
[0065] In the embodiments of this specification, before training the sign segmentation model using the second set of photos, the following may also be included:
[0066] The model parameters of the trained interest point classification model are transferred to the sign segmentation model to obtain the sign segmentation model with transferred parameters; the model parameters are used to determine whether there are interest points in the photo.
[0067] Training the sign segmentation model using a second set of photos can include:
[0068] The sign segmentation model with transferred parameters was trained using a second set of photos.
[0069] In this embodiment, the model parameters of the trained point of interest (POI) classification model are transferred to the sign segmentation model to obtain the sign segmentation model with transferred parameters. This means that the sign segmentation model inherits the model parameters of the POI classification model, thus reusing the model parameters of the POI classification model. This avoids training the original sign segmentation model using a second set of photos, or training the sign segmentation model from scratch using a second set of photos. This reduces the number of photos required in the second set and reduces the manual cost of marking the contours of the regions where POIs are located, while still enabling the sign segmentation model to accurately mark the contours of the regions where POIs are located.
[0070] For example, the processing efficiency of the sign segmentation model is approximately 11 FPS; it requires processing one high-resolution photo with a resolution of 896*896*3 pixels each time. For every 1% improvement in the accuracy of the sign segmentation model, 10,000 manually annotated photos need to be added to the second photo set. Generally, for one photo in the second photo set, the annotator needs 5 minutes of work and incurs a cost of 5 yuan; for one photo in the first photo set, the annotator needs 1 minute of work and incurs a cost of 1 yuan. In other words, the first photo set can be considered a low-cost photo set, and the second photo set can be considered a high-cost photo set. Figure 5 This diagram illustrates a comparison between the combined training cost of the sign segmentation model and the interest point classification model in one embodiment of this specification and the training cost of training the sign segmentation model alone. (See attached diagram.) Figure 5 Assuming the parameters of the trained interest point classification model are not transferred to the sign segmentation model, for a sign segmentation model with an accuracy of 80%, 80,000 manually labeled photos need to be added to the second photo set. Figure 5As shown in Figure A, the cost was 400,000 RMB. In this embodiment, as... Figure 5 As shown in Figure B, the model parameters of the trained point of interest (POI) classification model are transferred to the sign segmentation model. Training the POI classification model requires 50,000 photos at a cost of 50,000, while training the sign segmentation model requires 20,000 photos at a cost of 100,000, for a total cost of 150,000, far less than 400,000. Generally, when the first set of photos meets the requirements for training the POI classification model, the model parameters of the trained POI classification model are transferred to the sign segmentation model. The number of photos in the second set can be 10% to 40% of the number of photos in the first set, for example, 20%. This allows the sign segmentation model to accurately label the contours of the regions where POIs are located. This embodiment achieves the use of a relatively large, low-cost first set of photos and a small, high-cost second set of photos. By fine-tuning the model parameters of the sign segmentation model using the small second set of photos, significant cost savings are achieved.
[0071] In the embodiments of this specification, transferring the model parameters of the trained interest point classification model to the signboard segmentation model may include:
[0072] Determine the underlying model structure shared between the point of interest classification model and the signboard segmentation model;
[0073] The model parameters of the trained interest point classification model are transferred to the signboard segmentation model through the underlying model structure.
[0074] In this embodiment, the interest point classification model and the sign segmentation model can be pre-configured to share the same underlying model structure. When it is necessary to transfer the model parameters of the trained interest point classification model to the sign segmentation model, the shared underlying model structure between the two models is determined, and the model parameters of the trained interest point classification model are transferred to the sign segmentation model through the underlying model structure, which is very simple and convenient. The model parameter transfer process is essentially a copy and paste process, without changing the format of the model parameters.
[0075] In the embodiments of this specification, filtering the original photos using an interest point classification model may include:
[0076] The system uses an interest point classification model to record the labels of photos containing interest points in the original photos, and then removes photos whose labels are not recorded.
[0077] In this embodiment, the identification of photos containing points of interest in the original photos is recorded by the point of interest classification model. A table with two columns can be built in the database, one column storing the ID of the photo and the other column storing yes or no, so as to record whether each photo contains points of interest and remove photos whose identification is not recorded, which is very simple and convenient.
[0078] In the embodiments of this specification, filtering the original photos using an interest point classification model may include:
[0079] Intermediate features of the original photo are extracted using an interest point classification model, where the intermediate features are the image pixels of the original photo represented in vector form.
[0080] The original photos are scored based on the intermediate features of the original photos using an interest point classification model. Photos with scores below a preset threshold are removed, as they do not contain interest points.
[0081] In this embodiment, the intermediate features of the original photo are extracted by the point of interest classification model. The original photo is then scored based on the intermediate features of the original photo by the point of interest classification model. Photos with scores below a preset threshold are removed, which can yield a more accurate screening result, i.e., a more accurate target photo.
[0082] In the embodiments of this specification, marking the contour of the region where the point of interest is located on the target photo using a sign segmentation model may include:
[0083] The signboard segmentation model identifies the location of points of interest in the target photo based on the intermediate features of the target photo.
[0084] Mark the outline of the region containing the point of interest in the target image.
[0085] In this embodiment, the sign segmentation model identifies the region of interest (POI) in the target photo based on the intermediate features of the target photo. The outline of the POI region is then marked, which can generally be understood as marking the four corner points of the POI region's outline – a very simple and efficient process. Marking the outline of the POI region allows for the identification of only the text information contained within the marked area, avoiding interference from other areas and making the process more direct and efficient. Furthermore, the intermediate features of the target photo in this embodiment can be the intermediate features of the original photo extracted by the POI classification model in the aforementioned embodiment. This allows the intermediate features from the aforementioned embodiment to be used as input for this embodiment, fully utilizing the calculation results of the aforementioned embodiment to mark the outline of the POI region, avoiding redundant calculations, improving computational efficiency, and preventing the waste of computational power seen in the aforementioned embodiment.
[0086] Figure 2 This is a schematic diagram illustrating the architecture of a method for identifying points of interest in a photograph according to one embodiment of this specification. Both the point of interest classification model and the sign segmentation model can be convolutional neural networks. The point of interest classification model and the sign segmentation model can be implemented on the same AI platform or on different AI platforms. For details, see [link to documentation]. Figure 2 The point of interest (POI) classification model obtains the original photo and then extracts its intermediate features, which are the image pixels of the original photo represented by vectors. The POI classification model filters photos based on these intermediate features to obtain target photos containing POIs. The intermediate features of the target photos are stored in a Redis database. The sign segmentation model obtains intermediate features from the Redis database and marks the contours of the regions containing POIs in the target photos based on these features, obtaining segmented regions. The POI recognition module identifies the text information contained in the segmented regions and stores the recognized text information in a Redis database. Finally, the results of the text information stored in Redis are written to Elasticsearch (ES). In summary, this solution reduces the cost of manual annotation, increases the throughput of processed photos, optimizes the use of GPU computing resources, and improves the efficiency of updating POIs in maps. Generally, target photos (the intermediate features of the target photos) can be stored in a Redis database through a human-computer interface, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of a human-machine interface (HMI) for storing a target photo in a Redis database, as shown in one embodiment of this specification. Users can connect to the Redis server through the HMI, click the "Insert Row" button, and enter the identifier (order_id) and the photo's storage address (url). The identifier corresponds to the key in the HMI, and the storage address corresponds to the value in the HMI.
[0087] Figure 3 This is a schematic block diagram of a device for identifying points of interest (POIs) in a photograph according to one embodiment of this specification. Based on the same concept, the present invention also provides a device for identifying POIs in a photograph. The device may include a photograph acquisition module 501, a photograph filtering module 502, a POI segmentation module 503, and a POI recognition module 504. The photograph acquisition module 501 is used to acquire original photographs for updating POIs on a map; the photograph filtering module 502 is used to filter the original photographs using a POI classification model to obtain target photographs containing POIs; wherein, the POI classification model is used to determine whether there are POIs in the original photographs; the POI segmentation module 503 is used to mark the outline of the area where the POIs are located in the target photograph using a sign segmentation model to obtain the segmented area of the POIs in the target photograph; wherein, the sign segmentation model is used to segment the area where the POIs are located in the target photograph; the POI recognition module 504 is used to identify the POIs in the segmented area.
[0088] The present invention also provides a machine-readable storage medium having a machine-executable program and / or instructions stored thereon, wherein the machine-executable program and / or instructions, when executed by a processor, implement a method for identifying points of interest in a photograph according to any of the above embodiments.
[0089] It should be noted that the apparatus provided in this application corresponds one-to-one with the method provided in this application. Therefore, the apparatus also has similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus will not be repeated here. The various embodiments described above in this application can be combined arbitrarily.
[0090] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.
[0091] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0092] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0093] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.
[0094] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0095] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0096] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0098] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0099] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0100] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0101] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0102] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0103] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0104] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for identifying points of interest in a photograph, characterized in that, include: Obtain the original photos used to update points of interest on the map; The original photos are filtered using an interest point classification model to obtain target photos containing interest points; wherein, the interest point classification model is used to determine whether there are interest points in the original photos. The signboard segmentation model marks the contour of the region where the point of interest is located on the target photo based on the intermediate features of the original photo, thereby obtaining the segmented region of the point of interest on the target photo; wherein, the signboard segmentation model is used to segment the region where the point of interest is located on the target photo, and the intermediate features are extracted by the point of interest classification model, and the intermediate features are the image pixels of the original photo represented in vector form. Identify points of interest within the segmented region; The interest point classification model and the sign segmentation model share the underlying model structure. The model parameters of the trained interest point classification model are transferred to the sign segmentation model to obtain the sign segmentation model with transferred parameters. The sign segmentation model with transferred parameters is trained using a second set of photos.
2. The method according to claim 1, characterized in that, Before filtering the original photos using the point of interest classification model, the method further includes: Obtain the first set of photos containing annotation information for points of interest; The interest point classification model is trained using the first set of photos.
3. The method according to claim 2, characterized in that, Before marking the contours of the region containing the points of interest in the target photograph using the sign segmentation model, the method further includes: Obtain a second set of photos containing annotation information for the regions where points of interest are located; The sign segmentation model is trained using the second set of photos.
4. The method according to claim 3, characterized in that, The photos in the second photo set are at least a portion of the photos in the first photo set.
5. The method according to claim 3, characterized in that, The model parameters are used to determine whether there are points of interest in the photo.
6. The method according to claim 1, characterized in that, The filtering of the original photos using the point of interest classification model includes: The identification of photos containing points of interest in the original photos is recorded using a point of interest classification model, and photos whose identification is not recorded are removed.
7. The method according to claim 1, characterized in that, The filtering of the original photos using the point of interest classification model includes: The original photo is scored based on the intermediate features of the original photo using the point of interest classification model. Photos with scores below a preset threshold are removed, as they do not contain points of interest.
8. The method according to claim 7, characterized in that, The step of marking the contours of the regions containing points of interest in the target photo based on the intermediate features of the original photo using a sign segmentation model includes: The signboard segmentation model identifies the location of points of interest in the target photo based on the intermediate features of the target photo. The outline of the region containing the point of interest in the target photograph is marked.
9. A device for identifying points of interest in a photograph, characterized in that, include: The photo acquisition module is used to acquire the original photos for updating points of interest on the map. The photo filtering module is used to filter the original photos using an interest point classification model to obtain target photos containing interest points; wherein, the interest point classification model is used to determine whether there are interest points in the original photos. The point of interest segmentation module is used to mark the contours of the regions where the points of interest are located in the target photo based on the intermediate features of the original photo using a sign segmentation model, thereby obtaining the segmented regions of the points of interest in the target photo; wherein, the sign segmentation model is used to segment the regions where the points of interest are located in the target photo, and the intermediate features are extracted by the point of interest classification model, and the intermediate features are the image pixels of the original photo represented in vector form. An interest point recognition module is used to identify interest points in the segmented region; The interest point classification model and the sign segmentation model share the underlying model structure. The model parameters of the trained interest point classification model are transferred to the sign segmentation model to obtain the sign segmentation model with transferred parameters. The sign segmentation model with transferred parameters is trained using a second set of photos.
10. A machine-readable storage medium having stored thereon a machine-executable program and / or instructions, which, when executed by a processor, implement the method for identifying points of interest in a photograph according to any one of claims 1 to 8.