A problem map detection method, electronic device and system

Abstract

Embodiments of the present application provide a problem map detection method, an electronic device and a system, and relate to the technical field of image recognition. The method is applied to an electronic device and includes: acquiring a to-be-detected image; identifying target maps in the to-be-detected image, each target map including at least one target detection region; detecting each target map by using a problem map detection model to determine a violation type of the target detection region in each target map; and the violation type includes at least one of the following types: no violation, a first violation problem or a second violation problem. Through the technical solution provided by the embodiments, after the electronic device detects the target maps in the to-be-detected image, it can not only determine whether the target maps are problem maps, but also determine the specific violation problems of the target maps, obtain detailed map detection results, and has high detection efficiency.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a problem map detection method, electronic device and system. Background Technology

[0002] Problematic maps are maps containing errors such as misdrawing or omissions. These can be world maps, national maps, provincial maps, or city maps. Problematic maps are abundant on the internet, easily leading to misunderstandings about regional boundaries and causing numerous negative impacts. Currently, the detection of problematic maps is usually done manually, but manual detection is generally inefficient, especially when dealing with a large number of images. Therefore, there is an urgent need to provide a new method for detecting problematic maps. Summary of the Invention

[0003] This application provides a method, electronic device, and system for detecting problematic maps, which can improve the efficiency of detecting problematic maps.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, embodiments of this application provide a problem map detection method, the method comprising: acquiring an image to be detected; identifying all target maps in the image to be detected, each target map including at least one target detection region; detecting each target map using a problem map detection model, and determining the violation type of each target detection region in each target map.

[0006] The violation type includes at least one of the following: no violation, first violation issue, or second violation issue; no violation indicates that the corresponding target detection region has been identified and the drawing of the target detection region is compliant; first violation issue indicates that the corresponding target detection region has not been identified; second violation issue indicates that the corresponding target detection region has been identified, but the target detection region has a drawing error.

[0007] The problematic map detection method provided in this embodiment enables electronic devices to determine whether a target map in an image to be detected is a problematic map, as well as the specific violations that exist, and obtain detailed map detection results. This method has high detection efficiency and a better user experience.

[0008] In some embodiments, the types of target detection regions include: islands, boundary regions, and region insets. Based on this, the electronic device can detect different types of map regions, such as islands, boundary regions, and region insets.

[0009] In some embodiments, the first violation includes: missing islands, missing boundary areas, or missing area illustrations; the second violation includes: incorrect boundary drawing or different colors in local areas.

[0010] In some embodiments, acquiring the image to be detected specifically includes: acquiring the image to be detected and determining the target map. Through the method provided in this application embodiment, the electronic device can select the target map according to user instructions, meeting the user's need to detect different target maps.

[0011] In some embodiments, the problem map detection model is used to detect the target map and determine the violation type of each target detection region in each target map, including: detecting the image to be detected using the semantic segmentation network in the problem map detection model; and determining the violation type of each target detection region in each target map based on the detection results.

[0012] In some embodiments, when the target detection region is a boundary region, the image to be detected is detected using a semantic segmentation network in the problem map detection model. Determining the violation type of each target detection region in each target map based on the detection results specifically includes: identifying the first map mask where the boundary region is located using the semantic segmentation network; performing contour matching between the first map mask and a preset compliance map to determine the contour matching relationship; determining the key point coordinates of the boundary region in the first map mask based on the contour matching relationship and the coordinates of the boundary region in the compliance map; extracting the region image of the boundary region from the image to be detected based on the key point coordinates; and identifying the region image of the boundary region using a boundary recognition model to determine whether there is a boundary misdrawing problem in the boundary region.

[0013] In this embodiment, the electronic device uses a boundary recognition model to perform targeted detection on the region image of the boundary area, which can obtain relatively accurate detection results.

[0014] In some embodiments, when the target detection region is an island, the image to be detected is detected by the semantic segmentation network in the problem map detection model; determining the violation type of each target detection region in each target map based on the detection results specifically includes: identifying the second map mask where the island is located by the semantic segmentation network; if the second map mask is identified, it is determined that the island does not have an island omission problem; if the second map mask is not identified, it is determined that the island has an island omission problem.

[0015] In some embodiments, when the target detection region is an island, the image to be detected is detected by the semantic segmentation network in the problem map detection model; determining the violation type of each target detection region in each target map based on the detection results specifically includes: determining the key point coordinates of the island based on the second map mask; extracting the region image of the island from the image to be detected based on the key point coordinates of the island; and comparing the region image of the island with the target map in the image to be detected using a color recognition model to determine whether there is a problem of local color difference in the island.

[0016] In this embodiment, the electronic device uses a color recognition model to perform targeted detection on the regional image of the island to determine whether the island colors are drawn correctly, and the detection results are relatively accurate.

[0017] In some embodiments, when the target detection region is a region inset, the image to be detected is detected by the semantic segmentation network in the problem map detection model; determining the violation type of each target detection region in each target map based on the detection results specifically includes: identifying the third map mask where the region inset in the image to be detected is located through the semantic segmentation network; if the third map mask is identified, it is determined that there is no problem of missing region inset; if the third map mask is not identified, it is determined that there is a problem of missing region inset.

[0018] In some embodiments, each target map is detected using a problem map detection model to determine the violation type of each target detection region in each target map, including: detecting the image to be detected using a key point detection model in the problem map detection model; and determining the violation type of each target detection region in each target map based on the detection results.

[0019] In this embodiment, the key point detection model can quickly determine the location coordinates and response values ​​of key points in the target detection area, which helps to simplify the detection steps of the problem map and improve the detection rate.

[0020] In some embodiments, when the image to be detected includes multiple target maps, a keypoint detection model is used to detect the image to be detected and determine the violation type of each target detection region in each target map. This includes: identifying keypoints in all target detection regions of the multiple target maps using the keypoint detection model; classifying all keypoints according to their positional relationships to determine keypoints belonging to the same target map; determining target detection regions belonging to the same target map based on keypoints belonging to the same target map; and determining the violation type of the corresponding target detection region for each target map.

[0021] In some embodiments, the keypoint detection model in the problem map detection model is used to detect the image to be detected. Determining the violation type of each target detection region in each target map based on the detection results specifically includes: determining the response value of the target detection region using the keypoint detection model; when the target detection region is a boundary region and the response value of the keypoints in the boundary region is less than a threshold, it is determined that the boundary region has a problem of missing boundary region mapping; when the target detection region is an island and the response value of the keypoints in the island is less than a threshold, it is determined that the island has a problem of missing island mapping; when the target detection region is a region illustration and the response value of the keypoints in the region illustration is less than a threshold, it is determined that the region illustration has a problem of missing region illustration mapping.

[0022] In some embodiments, when the target detection area is a boundary area, the problem map detection model is used to detect each target map and determine the violation type of each target detection area in each target map. Specifically, this includes: when the response value of the boundary area is greater than or equal to a threshold, determining the key point coordinates of the boundary area using a key point detection model; based on the key point coordinates, extracting the region image of the boundary area from the image to be detected; and recognizing the region image of the boundary area using a boundary recognition model to determine whether there is a boundary misdrawing problem in the boundary area.

[0023] In some embodiments, when the target detection area is an island, the problem map detection model is used to detect each target map and determine the violation type of each target detection area in each target map. Specifically, this includes: when the response value of the key point of the island is greater than or equal to a threshold, the key point detection model is used to determine the coordinates of the key point of the island; based on the key point coordinates of the island, the region image of the island is extracted from the image to be detected; and the color recognition model is used to compare the region image of the island with the target map in the image to be detected to determine whether there is a problem of local color difference in the island.

[0024] In some embodiments, cropping a corresponding region image based on the key point coordinates of the target detection region includes: determining the pixel area of ​​the region image based on the pixel area of ​​the image to be detected; determining the width and height of the target detection region based on a preset aspect ratio of the target detection region and the pixel area of ​​the region image; and cropping a region image of the target detection region from the image to be detected based on the key point coordinates of the target detection region and the width and height of the target detection region.

[0025] In some embodiments, cropping a corresponding region image based on the key point coordinates of the target detection region includes: determining the average distance between key points of multiple target detection regions on the same target map; determining the width and height of the target detection region based on a preset aspect ratio of the target detection region and the average distance; and cropping a region image of the target detection region from the image to be detected based on the key point coordinates of the target detection region and the width and height of the target detection region.

[0026] Secondly, embodiments of this application provide a problem map detection system, including a client and a server; the client is configured to send an image to be detected to the server; the server is configured to identify all target maps in the image to be detected, each target map including at least one target detection area; detect each target map using a problem map detection model to obtain detection results, the detection results including: the violation type of each target detection area in each target map; and send the detection results to the client.

[0027] The violation type includes at least one of the following types: no violation, first violation issue, or second violation issue; no violation indicates that the corresponding target detection area has been identified and the drawing of the target detection area is compliant; first violation issue indicates that the corresponding target detection area has not been identified; second violation issue indicates that the corresponding target detection area has been identified, but the target detection area has a drawing error.

[0028] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the problem map detection method as shown in the first aspect above.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the problem map detection method as described in the first aspect above.

[0030] Fifthly, embodiments of this application provide a computer program product, which includes a program that, when run by an electronic device, causes the electronic device to perform the problem map detection method as shown in the first aspect above.

[0031] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0032] Figure 1 A compliant map of Province A provided for embodiments of this application;

[0033] Figure 2 An illegal map of Province A provided in this application embodiment;

[0034] Figure 3 This is a schematic flowchart of the problem map detection method provided in the embodiments of this application;

[0035] Figure 4 This is a schematic diagram of the preprocessing of an image to be detected provided in one embodiment of this application;

[0036] Figure 5 This is a schematic diagram of the preprocessing of an image to be detected provided in another embodiment of this application;

[0037] Figure 6 This is a schematic diagram of an image to be detected provided in one embodiment of this application;

[0038] Figure 7 This is a schematic flowchart of a problem map detection method provided in one embodiment of this application;

[0039] Figure 8 This is a schematic diagram of the image processing procedure to be detected provided in one embodiment of this application;

[0040] Figure 9 This is a schematic diagram of position coordinates provided in one embodiment of this application;

[0041] Figure 10 This is a schematic diagram illustrating the sampling point offset angle as provided in one embodiment of this application;

[0042] Figure 11 This is a schematic diagram illustrating the determination of sampling point offset provided in one embodiment of this application;

[0043] Figure 12 This is a schematic diagram illustrating the matching relationship between the outline of a target map mask and the outline of a compliant map, provided in one embodiment of this application.

[0044] Figure 13 This is a region image of different key detection areas in one embodiment of this application;

[0045] Figure 14 This is a schematic diagram illustrating the identification process for regional color discrepancies, provided in one embodiment of this application.

[0046] Figure 15 This is a schematic diagram of the detection results provided in one embodiment of this application;

[0047] Figure 16A This is a schematic diagram of an image to be detected provided in another embodiment of this application;

[0048] Figure 16B This is a schematic diagram of the detection results provided in another embodiment of this application;

[0049] Figure 17 This is a schematic diagram of an image to be detected provided in another embodiment of this application;

[0050] Figure 18 This is a schematic flowchart of a problem map detection method provided in another embodiment of this application;

[0051] Figure 19 This is a schematic diagram of the image processing procedure to be detected provided in another embodiment of this application;

[0052] Figure 20 This is a schematic diagram of key points provided in one embodiment of this application;

[0053] Figure 21 This is a schematic diagram of a key point classification process provided in one embodiment of this application;

[0054] Figure 22 These are region images of different key detection areas provided in the embodiments of this application;

[0055] Figure 23 This is a schematic diagram of the detection results provided in another embodiment of this application;

[0056] Figure 24 This is a schematic structural diagram of an electronic device provided in one embodiment of this application;

[0057] Figure 25 This is a flowchart of the problem detection system provided in the embodiments of this application;

[0058] Figure 26 This is a schematic diagram illustrating the setting of the problem map detection strategy provided in the embodiments of this application. Detailed Implementation

[0059] The technical solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0060] Problematic maps are maps containing errors such as incorrectly drawn boundaries, missing islands, inconsistent coloring in certain areas, and missing regional insets. These can be world maps, national maps, provincial maps, or city maps. For example, consider a map of province A. Figure 1 For a compliant map of Province A, Figure 2 This is a map of Province A that violates regulations (also known as a problematic map). Figure 2 In the problematic map shown, there are issues with the boundary drawing of regions A and B, a local color discrepancy in region C (i.e., the color of region C is different from the colors of other regions on the map of province A), and a missing regional illustration in region D.

[0061] Problematic maps are prevalent on the internet, easily leading to misunderstandings about regional maps and causing numerous adverse effects. Currently, the detection of problematic maps is usually done manually, but manual detection is generally inefficient, especially when there are a large number of images to be detected, making it difficult to complete the map detection task efficiently. Therefore, this application provides a method for detecting problematic maps to improve the efficiency and accuracy of detection.

[0062] The problem map detection method provided in this application can be applied to electronic devices such as terminal devices and servers. Terminal devices include desktop computers, mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. This application does not impose any restrictions on the specific type of terminal device.

[0063] Figure 3 This is a schematic flowchart illustrating the problematic map detection method provided in this application embodiment. It involves using an electronic device to detect map violations through a problematic map detection model, thereby identifying map violations. It should be noted that in this application embodiment, the problematic map detection model includes a series of sub-models, such as one or more models like a target map recognition model, a key point detection model, a semantic segmentation network, a boundary recognition model, a color recognition model, and a regression recognition model. The method specifically includes the following steps.

[0064] S301, The electronic device acquires the image to be detected.

[0065] In this embodiment, the content of the image to be detected may include one or more maps, such as world maps, national maps, provincial maps, or city maps, or one or more local areas of the above types of maps. In this embodiment, "multiple" refers to two or more.

[0066] S302, The electronic device preprocesses the image to be detected to obtain a preprocessed image.

[0067] Optionally, after acquiring the image to be detected, the electronic device can preprocess it to convert it into a square image of the target size for subsequent image recognition. For example, the target size can be 512*512 pixels, 416*416 pixels, or 224*224 pixels, etc., and this embodiment does not impose any limitations on this. Taking a 512*512 pixel image as an example, 512 pixels refers to the number of pixels on the horizontal or vertical edge of the image to be detected.

[0068] Taking a target size of 512*512 pixels as an example, when the image to be detected is square, the electronic device can directly scale it (i.e. shrink or enlarge it) to process it into a square image of 512*512 pixels.

[0069] When the image to be detected is, for example Figure 4 In the case of the rectangular image shown in (a), one of its horizontal and vertical sides must be a long side and the other a short side. If the electronic device directly scales the long side to 512 pixels and the short side to 512 pixels, the pre-processed image will be distorted (e.g., Figure 4 (as shown in (b)).

[0070] Therefore, for a rectangular image to be detected, the electronic device can scale its long and short sides proportionally. When the long side is scaled to 512 pixels, the length of its short side will necessarily be less than 512 pixels. At this point, the electronic device can use black pixels to supplement the scaled image to make the short side 512 pixels long, thus obtaining a 512*512 pixel preprocessed image to avoid image distortion.

[0071] For example, for Figure 5 The rectangular image to be detected shown in (a) can be processed into the method provided in this embodiment. Figure 5 The preprocessed image of 512*512 pixels shown in (b) is shown in the figure.

[0072] S303, Electronic device identifies whether there is a target map in the pre-processed image.

[0073] The target map refers to the object of this map detection task, such as a world map, a map of province A, a map of province S, etc. This target map is preset and usually varies depending on the specific detection task.

[0074] It should be noted that during the problematic map detection process, the electronic device only identifies and detects the target map, and does not detect other maps in the preprocessed image. For example, when the preprocessed image includes maps of Province A, Province B, and Province C, if the target map for this detection task is the map of Province A, the electronic device will only detect the drawing problems of the map of Province A, and will not detect the maps of Provinces B and C.

[0075] In this embodiment, the electronic device can use a target map recognition model or a map classification model to identify whether the preprocessed image contains a target map. If the preprocessed image contains a target map, the next step S304 is executed. If the preprocessed image does not contain a target map, the detection task for that preprocessed image ends. It should be noted that the target map recognition model is preset, and the target maps that the target map recognition model can recognize will vary depending on the pre-trained data. Furthermore, one target map recognition model can recognize one or more target maps.

[0076] S304, When the preprocessed image includes a target map, the electronic device identifies the target detection area in the target map.

[0077] The target detection region refers to a specific area in a target map. It is typically an area prone to errors during the target map creation process and is predetermined. The target detection regions are usually different for different target maps. For example, see... Figure 1 and Figure 2 As shown, areas A through D in the map of Province A are prone to drawing errors. Therefore, areas A through D can be designated as the target detection area for the map of Province A, and their drawing can be checked for compliance, without checking other areas in the map of Province A.

[0078] There are several methods for electronic devices to identify target detection regions. One possible approach is to identify target detection regions based on semantic segmentation. Another possible approach is to use a keypoint detection model to identify target detection regions. See the description below for details; this embodiment will not elaborate further.

[0079] S305 If the target detection area is not identified, the electronic device determines that there is a first violation problem in the target detection area.

[0080] S306, if a target detection area is identified, the electronic device determines the area image of the target detection area and determines whether a second violation problem exists in the target detection area based on the area image.

[0081] Different types of target detection regions typically exhibit different violations. This embodiment categorizes violations into first violations and second violations.

[0082] The first violation issue refers to the problem of missing target detection areas, including missing islands, missing region insets, and missing boundary regions. The first violation issue typically differs for different types of regions. For example, the first violation issue for islands (such as region C) is usually missing islands, while the first violation issue for region insets (such as region D) is usually missing region insets.

[0083] The second violation refers to the problem where the target detection area exists, but the boundary or color is drawn incorrectly, including misdrawn boundaries and color discrepancies in local areas. The second violation typically differs for different types of areas. For example, the second violation commonly found in map edge areas (such as area A and area B) is misdrawn boundaries, while the second violation commonly found in islands (such as area C) is missing area insets.

[0084] Therefore, if the electronic device fails to identify the target detection area, it is determined that the target detection area has a first violation problem. Once the electronic device identifies the target detection area, it is necessary to further analyze the area image to determine whether a second violation problem exists in that area.

[0085] Region images are typically regular-shaped local images within a preprocessed image, such as rectangles, squares, circles, or ellipses, and include map information about the target detection area. For different types of target detection regions, electronic devices can use different recognition models to identify their region images, thereby specifically detecting frequently occurring problems in that region.

[0086] For example, as shown in Table 1, for regions located at the edge of the map (such as region A and region B), only the boundary recognition model is used to detect whether there are boundary misdrawing issues, without detecting whether there are missing islands or local color discrepancies. For islands (such as region C), a color recognition model can be used to identify whether the island's color is consistent with the overall map color.

[0087] Table 1 Recognition Model Table

[0088]

[0089] It should be noted that in this embodiment, each recognition model is pre-trained for images with specific characteristics and is only used to recognize one or more specific images. For example, for a boundary recognition model, if it has been trained on a large number of image training samples containing compliant region A, then it will have the ability to recognize the boundary drawing of region A. If it has been trained on a large number of training samples containing images of compliant regions A and B, then it will have the ability to recognize the boundary drawing of both regions A and B.

[0090] In summary, in the problem map detection method provided in this embodiment, the electronic device extracts the target detection area image from the image to be detected, and then, according to the different types of target detection areas, uses the corresponding image recognition model to perform image recognition on the image of that area separately, thereby determining whether there is a violation problem in that area. Compared with the overall recognition of the image to be detected and manual problem map detection, the method provided in this embodiment has the characteristics of returning detailed results, fast speed, and high accuracy.

[0091] Furthermore, using the method provided in this embodiment, after recognizing an image, the electronic device can not only determine whether it is a problematic map, but also identify the specific violations it contains, obtaining detailed map detection results and providing a better user experience. For example, for Figure 2 The image to be detected shown can be identified by the electronic device as a problematic map. Furthermore, regions A and B of the map have incorrect boundary drawing, region C has local color discrepancies, and region D has omissions in the subplot.

[0092] The image below is to be detected. Figure 6 Taking the image and target map as a map of province A, and the target detection area as including region A, region B, region C, and region D as an example, the problem map detection method provided in this application embodiment will be described by way of example.

[0093] Figure 7 This is a schematic flowchart illustrating a problem map detection method provided in one embodiment of this application. It involves an electronic device identifying target detection regions based on a semantic segmentation network, determining the regional image of the target detection regions, and identifying violations in each target detection region. Specifically, it includes the following steps.

[0094] S701, The electronic device acquires the image to be detected.

[0095] In this embodiment, the image to be detected (i.e. Figure 6 or Figure 8 The image shown in (a) is rectangular and contains only one map of Province A.

[0096] S702, the electronic device preprocesses the image to be detected to obtain a preprocessed image.

[0097] In this embodiment, the electronic device... Figure 8 After preprocessing the image to be detected shown in (a), we can obtain Figure 8 The preprocessed image shown in (b) is shown in the example. In one example, the preprocessed image is a square image with a size of 512*512 pixels.

[0098] S703, the electronic device identifies whether the preprocessed image includes a map of province A.

[0099] Optionally, the electronic device can use a preset target map recognition model to identify the preprocessed image and determine its response value. If the response value is greater than or equal to a threshold (e.g., 0.5), it indicates that the preset image includes the target map. If the response value is less than the threshold, it indicates that the preset image does not include the target map.

[0100] For example, target map recognition models in the context of... Figure 8 After the preprocessed image shown in (b) is identified, a response value of 0.99 is detected. Since this response value is greater than the threshold of 0.5, the electronic device determines that the preprocessed image includes a map of province A and proceeds to the next step S704.

[0101] S704, the electronic device identifies target detection regions in a preprocessed image through a semantic segmentation network.

[0102] In this embodiment, see Figure 8 As shown in (b), regions A and B are located in the first part of the map of province A, region C is located in the second part of the map of province A, and region D is located in the third part of the map of province A. These three parts are not connected to each other. Therefore, when performing semantic segmentation, the electronic device can determine the existence of a target detection region by identifying the presence of these three parts. If the electronic device detects the first part, it considers region A and region B to have been identified. If the electronic device detects the second part, it considers region C to have been identified. If the electronic device detects the third part, it considers region D to have been identified.

[0103] In this embodiment, the electronic device can perform semantic segmentation on the preprocessed image using a U-shape network (U-Net) or other classic semantic segmentation networks (such as DeepLab Series Network, Deep Lab V3) to identify the first, second, and third parts of the map of Province A.

[0104] It should be noted that the map of Province A also includes other parts besides the first, second, and third parts. For the sake of convenience, these parts are not all listed here in this embodiment.

[0105] S705, if the target detection area is not identified, the electronic device determines that there is a first violation problem in the target detection area.

[0106] For regions A and B, since the first part they belong to is the main part of the map of province A, if the electronic device detects that the preprocessed image includes the map of province A, it indicates that the first part exists. Therefore, in this embodiment, regions A and B generally will not have the first violation problem.

[0107] For region C, if the electronic device does not recognize region C in the preprocessed image, then region C is determined to have the island omission problem in the first violation problem.

[0108] For region D, if the electronic device does not recognize region D in the preprocessed image, then region D is determined to have the omission of region inset in the first violation problem.

[0109] S706 If a target detection area is identified, the electronic device determines the map mask corresponding to the target detection area.

[0110] S707, an electronic device recognizes a map mask and determines the region information of the target detection area, which includes key point information or region inset size.

[0111] In this embodiment, a key point refers to a representative point in the target detection region, such as the center point. Key point information includes its name and coordinate position. The name of the key point indicates the target detection region it corresponds to. In this embodiment, for ease of understanding, the key point in region A can be named "Point A," the key point in region B can be named "Point B," and so on. The coordinate position refers to the coordinate location of the key point in the preprocessed image.

[0112] It should be noted that, see Figure 9 As shown, in this embodiment, the top-left corner of the preprocessed image is defined as the origin (0,0), the direction passing through the origin and moving horizontally to the right along the preprocessed image is defined as the positive direction of the X-axis, and the direction passing through the origin and moving vertically downwards along the preprocessed image is defined as the positive direction of the Y-axis. Based on this, in Figure 9 In the preprocessed image shown, the coordinates of the key points in region A are: .

[0113] The methods for determining the regional information of region A, region B, region C, and region D are explained below.

[0114] (1) Region A, Region B

[0115] Since regions A and B are located on the edge of the first part of the map of province A, the electronic device determines the key point information of regions A and B by recognizing the outline of the first part. The details are shown below.

[0116] First, the electronic device determines a first map mask for the first part. Optionally, after performing semantic recognition on the first part of the preprocessed image, the electronic device can obtain the response value of each pixel in the preprocessed image. This response value is typically between 0 and 1; the larger the response value of a pixel, the greater the probability that the pixel is located in the first part. Based on this, the electronic device can binarize the semantic segmentation result according to whether the response value of the pixel is greater than a threshold, generate a first feature map corresponding to the first part, and determine the first map mask based on the first feature map.

[0117] For example, electronic devices Figure 8 After semantic segmentation of the preprocessed image shown in (b), and determining the response value of each pixel, all pixels with response values ​​less than the threshold of 0.5 are identified as non-target pixels (i.e., the pixel does not belong to the first part of the map of province A), and all pixels with response values ​​greater than or equal to 0.5 are identified as target pixels (i.e., the pixel belongs to the first part of the map of province A). This allows for the generation of, for example... Figure 8 The first feature map is shown in (c1). In the first feature map, the largest connected region (i.e., the largest white region) of the target pixel is the first map mask.

[0118] Secondly, the electronic device matches the first map mask with the outline of the compliant map to determine the outline matching relationship. This process specifically includes the following steps a~e.

[0119] a. The electronic device extracts the contour points of the first map mask and samples these contour points at equal intervals to determine N sampling points. The distance between any two contour points along the contour line is equal. N is a preset value, such as N=100, N=200, N=500, etc. Figure 10 Taking the local contour line shown as an example, this local contour line is Figure 8 (c1) shows a portion of the contour line of the first map mask. On this contour line, each hollow and solid origin represents a contour point of the first map mask. If a sampling point is determined every two contour points, then... Figure 10 As shown, among all contour points, the hollow origin is the sampling point.

[0120] b. The electronic device determines the total offset angle of each sampling point relative to its neighboring sampling points. The number of neighboring sampling points is usually an even number, such as 4, 6, 8, etc. This embodiment does not limit this.

[0121] The following is combined Figure 10 The sampling point is , The nearest sampling point is , , and Taking this as an example, the process of determining the total offset angle of the sampling point will be explained.

[0122] See Figure 10 As shown, sampling points The baseline is The baseline is parallel to the horizontal or vertical edge of the preprocessed image, and the baselines of all sampling points are parallel. and The line in question is , and The line in question is , and The line in question is , and The line in question is .in, and The included angle That is Compared to The deflection angle; and The included angle That is Compared to The deflection angle; and The included angle That is Compared to The deflection angle; and The included angle That is Compared to The deflection angle.

[0123] It should be noted that, in this embodiment, a certain straight line (e.g.) ) and baseline The included angle refers to the baseline. Rotate counterclockwise until it aligns with the straight line (e.g.) The minimum angle of rotation required when they coincide.

[0124] In conclusion, The offset angle relative to its neighboring sampling points includes , Total offset angle relative to its neighboring sampling points , Also known as The eigenvalues ​​of a point.

[0125] c. Electronic devices use a standardization processing formula to standardize the feature values ​​of each sampling point, processing the feature values ​​to values ​​between 0 and 1, in order to reduce the computational data.

[0126] In this embodiment, the standardization formula is:

[0127]

[0128] in, These are the standardized feature values. for Unprocessed eigenvalues It is the minimum value among the feature values ​​of all sampling points. It is the maximum value among the feature values ​​of all sampling points.

[0129] By arranging the standardized feature values ​​of all sampled points in order, the feature vector of the target map mask can be obtained. .

[0130] Similarly, referring to the above The method for determining this method also allows electronic devices to determine the feature vector of the compliant map corresponding to the target map. .in, Sampling points in the compliant map The corresponding eigenvalues.

[0131] d. Electronic equipment utilization right Offset correction is performed on each sampling point.

[0132] Optionally, the electronic device can use a calibration formula to determine The offset of the first map mask corresponding to the range. In this embodiment, the correction formula is as follows:

[0133]

[0134] in, And it is an integer. Take in sequence Take in sequence , k The total number of sampling points. j This indicates the size of the sliding window. In this embodiment, the sliding window refers to the maximum integration limit (i.e., ...). ) and the lower limit of integration (i.e. The difference can also be used to represent the number of sampling points involved in the integration operation, and the size of the sliding window. j It is preset and can be 6, 8, 10, etc., but this embodiment does not limit it.

[0135] by And it is an integer. For example, electronic devices need to calculate when When equal to -4, -3, -2, -1, 0, 1, 2, 3, and 4, The integral value. See [reference]. Figure 11 As shown, this integral value is used to represent and The smaller the area, the greater the difference. and The higher the degree of overlap, the better. Among these integral values, the smallest integral value corresponds to... That is Medium sampling point The offset.

[0136] By gradually moving the sliding window (that is, simultaneously increasing or decreasing the upper and lower limits of the integral in the correction formula) and repeating the above offset correction process, the offset can be determined. The offset of each sampling point in the map is used to determine the correspondence between the sampling points of the target map mask and the compliant map. This involves matching the outline of the target map mask with the outline of the compliant map. (See also...) Figure 12 As shown. Where M is and The perspective transformation matrix between them.

[0137] e. The electronic device determines the key point information of region A and region B based on the contour matching relationship between the first map mask and the compliant map.

[0138] Since regions A and B are pre-defined, they have corresponding sampling point information within the compliance map. Therefore, based on the contour matching relationship between the target map and the compliance map... By combining the sampling point information of region A and region B in the compliant map, the key points corresponding to region A and region B in the target map can be determined, and then the coordinates of the key points can be determined.

[0139] (2) Region C

[0140] In some embodiments, referring to the method for determining the first feature map of the first portion shown above, the electronic device can determine the second feature map corresponding to the second portion (e.g., Figure 8 (c2)). In the second feature map, the white area is the second map mask. Since region C occupies the entire area of ​​the second part, the center position of the second map mask can be determined as the key point of region C, and the coordinates of this center position are the coordinates of the key point of region C.

[0141] In other embodiments, because maps differ from ordinary images, the relative positional relationships between different regions of a specific map are predetermined, as are the relative positional relationships of key points within those regions. For example, for a map of province A, the relative positional relationships between region A, region B, and region C are predetermined, as are the relative positional relationships between its key points (i.e., points A, B, and C). Based on this, the electronic device can determine the coordinates of point C after determining the coordinates of point A and point B, according to the relative positional relationships of points A, B, and C.

[0142] (3) Region D

[0143] Regarding the third part, referring to the method for determining the first feature map of the first part shown above, the electronic device can also determine the third feature map corresponding to the third part (e.g., Figure 8 (c3)). In the third feature map, the white area is the third map mask. Since region D occupies the entire area of ​​the third part, the center of the third map mask is the key point of region D, and the coordinates of this center are the coordinates of the key point of region D.

[0144] Furthermore, for region D, the electronic device can also identify the dimensions of the outer rectangle of the third map mask and determine them as the dimensions of the region inset. That is, the width and height of the outer rectangle are determined as the width and height of the region inset.

[0145] In summary, for Figure 8 The preprocessed image shown in (b) can be used by the electronic device to determine the key point information shown in Table 2 using the key point information determination method shown in S706.

[0146] Table 2 Key Point Information

[0147]

[0148] It should be noted that in this embodiment, each pixel is used as a unit length when determining coordinates and length. For example, when the length of a line is 53, it means that the length of the line is 53 units.

[0149] S708, the electronic device determines the region image corresponding to the target detection region based on key point information.

[0150] In this embodiment, according to the detection task requirements, the electronic device needs to determine the regional images of regions A and B to detect whether they have the boundary misdrawing problem in the second violation issue. It also needs to determine the regional image of region C to detect whether it has the local area color difference problem in the second violation issue. Region D typically does not have the second violation issue; therefore, it is not necessary to determine the regional image of region D.

[0151] Optionally, the electronic device can determine the image area of ​​the target detection region based on the area S of the target map. .For example, .in, For example, the default value. ,or wait.

[0152] For each target detection region, the electronic device pre-determines the aspect ratio of the region's image. For example, for regions A, B, and C, the aspect ratios of their respective region images are [1 / 1.5, 1 / 1.9], [1 / 1.5, 1 / 1.2], and [1 / 1.625, 1 / 2]. It should be noted that these aspect ratios are determined based on the terrain characteristics of different target detection regions, and the aspect ratio of the region images is similar to that of the target detection regions.

[0153] Electronic devices detect the area of ​​the target region. The length and width of the region image can be determined using the following formula.

[0154]

[0155] in, Let A be the length of the image in region A. Let A be the width of the image in region A, and , The area of ​​the region in the image where the target is detected is denoted as .

[0156] Given that the pixel area of ​​the target map is S=96993, Taking the target detection regions as regions A, B, and C, and the preset aspect ratios of the region images of regions A, B, and C as [1 / 1.5, 1 / 1.9], [1 / 1.5, 1 / 1.2], and [1 / 1.625, 1 / 2] respectively, the length and width of the region images of regions A, B, and C can be determined to be [65, 51], [65, 81], and [60, 49] respectively using the above formula.

[0157] In summary, electronic devices can obtain, for example, the regional image information shown in Table 3, which includes the map where each target detection region is located, the name of the target detection region, the name of the key point, the coordinates of the center position of the regional image (i.e., the coordinates of the key point), and the correspondence between the regional image size and the target detection region.

[0158] Table 3 Regional Image Information

[0159]

[0160] Based on the correspondence in Table 3, the electronic device can determine the region images of each target detection area in the image to be detected. For example, for... Figure 8 The image to be detected shown in (a) allows the electronic device to determine the region images of regions A, B, and C as follows: Figure 13 (a) Figure 13 (b) and Figure 13 (c) in the middle.

[0161] S709, the electronic device determines whether a second violation exists in the corresponding target detection area based on the area image.

[0162] In this embodiment, depending on the type of target detection region, as shown in Table 1, the electronic device can use different image recognition models to identify the second violation problem in the corresponding region image.

[0163] For example, for Figure 13 In the region image of region A shown in (a), since the electronic device determines its violation classification score to be 0.970 through the boundary recognition model, which is greater than the threshold of 0.5, it is determined that there is no boundary misdrawing problem in the second violation problem in region A.

[0164] For example, for Figure 13 In the region image of region B shown in (b), since the electronic device can determine its violation classification score of 0.977 through the boundary recognition model, which is greater than the threshold of 0.5, it is determined that there is no boundary misdrawing problem in the second violation problem in region B.

[0165] For example, regarding the Figure 13 In the region image of region B shown in (b), since the electronic device can determine its violation classification score of 0.989 through the color recognition model, which is greater than the threshold of 0.5, it is determined that there is no problem of local regional color difference.

[0166] It should be noted that when the color recognition model identifies whether there are localized color variations in region C, it needs to combine local color features (i.e., the color features of region C) and global color features (i.e., the overall color features of the target map in the image to be detected) for judgment. Therefore, see [link to relevant documentation]. Figure 14As shown, the color recognition model first needs to acquire the region image of region C and the image to be detected, respectively. Then, convolution (Conv) and global average pooling (GAP) are performed on the region image of region C and the image to be detected, respectively, to obtain the color feature information of region image C and the image to be detected. Finally, convolution, global average pooling, and fully connected (FC) processing are applied to the color feature information of region image C and the image to be detected to output a violation classification score. When the violation classification score is greater than a threshold, it is determined that region C does not have a local color discrepancy problem; when the violation classification score is less than the threshold, it is determined that region C has a local color discrepancy problem.

[0167] Based on the above S701-S709, the electronic device completes Figure 8 After recognizing the image to be detected, as shown in (a) above, a detection result is generated. For example, the detection result is shown below:

[0168] {

[0169] "success_code":200;

[0170] "error_message":"success";

[0171] "suggestion": "normal"; #Final review result

[0172] “map_classify_results”:[ # Target map classification results]

[0173] {

[0174] “label”:“others”,

[0175] "confidence": 0.0;

[0176] }

[0177] {

[0178] “label”:“map”,

[0179] "confidence":1.0;

[0180] }

[0181] ],

[0182] “regional_location”:[ [

[0184] {

[0185] “label”:“A_area”, #area category

[0186] “location”:[ #keypoint coordinates]

[0187] 240, 180

[0189] ],

[0190] "suggestion": "normal", # Judgment of violation classification results

[0191] "type":0, #Type of violation

[0192] “normal_score”: 0.970 # Score for violation category

[0193] },

[0194] {

[0195] “label”:“B_area”, #area category

[0196] “location”:[ #keypoint coordinates]

[0197] 110, 340

[0199] ],

[0200] "suggestion": "normal", # Judgment of violation classification results

[0201] "type":0, #Type of violation

[0202] “normal_score”: 0.977 # Score for violation category

[0203] },

[0204] {

[0205] “label”:“C_area”, #area category

[0206] “location”:[ #keypoint coordinates]

[0207] 377, 371

[0209] ],

[0210] "suggestion": "normal", # Judgment of violation classification results

[0211] "type":0, #Type of violation

[0212] “normal_score”: 0.989 # Score for violation of category

[0213] } ] ]

[0216] “Add_area_box”:{ # Area plotting detection box

[0217] "confidence": 0.922

[0218] “bounding_box”:[

[0219] 480,

[0220] 395,

[0221] 53, 79 ]

[0224] "suggestion": "normal", # Judgment of violation classification results

[0225] "type":0, #Type of violation

[0226] },

[0227] "version": 1.0

[0228] }

[0229] It should be noted that in the various embodiments of this application, when the violation type (type) of the target detection region is "0", it means that the target detection region is compliant. When the violation type (type) of the target detection region is "1", it means that the target detection region has a boundary misdrawing problem. When the violation type (type) of the target detection region is "2", it means that the target detection region has an island omission problem. When the violation type (type) of the target detection region is "3", it means that the target detection region has a local area color difference problem. When the violation type (type) of the target detection region is "4", it means that the target detection region has an omission of the region illustration problem.

[0230] Since the electronic device detects that no violations exist in any of the target detection regions on the image to be inspected, it considers the map to be a compliant map and displays the detection result on the image to indicate that the map is compliant. For example, see... Figure 15As shown, the electronic device can highlight the detected regions A, B, C, and D on the image to be inspected, and display the text information "Compliance Map: Region A is compliant, Region B is compliant, Region C is compliant, Region D is compliant".

[0231] In other embodiments, the image to be detected is Figure 16A The image shown, taking a target map of province A and target detection areas including regions A, B, C, and D as an example, demonstrates how the problem map detection method provided in this application can obtain the following results: Figure 16B The detection results are shown below. These results indicate that the target map in the image is a problematic map. Specifically, region A has a misdrawn boundary, region B has a misdrawn boundary, region C has a local color discrepancy, and region D has an inset omission.

[0232] The image below is to be detected. Figure 17 The image shown is a map of province A, and the target detection area includes region A, region B, region C, and region D. This example illustrates the problem map detection method provided in this application.

[0233] Figure 18 This is a schematic flowchart illustrating a problem map detection method provided in another embodiment of this application. It involves an electronic device identifying target detection areas using a key point detection model, thereby determining the regional image of the target detection areas and identifying violations in each target detection area. Specifically, it includes the following steps.

[0234] S1801, The electronic device acquires the image to be detected.

[0235] In this embodiment, the image to be detected is, for example... Figure 19 (a) or Figure 17 The image shown is rectangular and contains two maps of Province A, namely... Figure 1 peacefully Figure 2 .

[0236] S1802, The electronic device preprocesses the image to be detected to obtain a preprocessed image.

[0237] In this embodiment, the electronic device... Figure 19 After preprocessing the image to be detected shown in (a), we can obtain Figure 19 The preprocessed image shown in (b) is shown in the example. In one example, the preprocessed image is a square image with a size of 512*512 pixels.

[0238] S1803, Electronic equipment identifies whether a pre-processed image includes a map of province A.

[0239] Optionally, the electronic device can use a preset target map recognition model to identify the preprocessed image and determine its response value. If the response value is greater than or equal to a threshold, it indicates that the preset image includes the target map. If the response value is less than the threshold, it indicates that the preprocessed image does not include the target map.

[0240] For example, target map recognition models in the context of... Figure 19 After the preprocessed image shown in (b) is identified, a response value of 0.98 is detected. Since this response value is greater than the threshold of 0.5, the electronic device determines that the preprocessed image includes a map of province A and proceeds to the next step S1804.

[0241] S1804, the electronic device uses a key point detection model to identify target detection regions in a preprocessed image.

[0242] In this embodiment, the electronic device uses a high-resolution network (HR net) keypoint detection model to identify keypoints in the target detection region. If the response value of a keypoint is less than a threshold, it is considered that no target detection region has been identified. If the response value of a keypoint is greater than or equal to the threshold, it is considered that a target detection region has been identified.

[0243] For example, targeting Figure 19 The preprocessed image shown in (b) shows the key point information of each target detection region identified by the electronic device, as shown in Table 4. For each identified target detection region, the electronic device also determines the name and coordinates of its key points. (See Table 4 for details.) Figure 1 Regions A, B, C, and D, and the land Figure 2 The response values ​​of key points in regions A, B, and C are all greater than the threshold of 0.5. Therefore, the electronic device determines that it has identified two regions A, two regions B, two regions C, and one region D.

[0244] Table 4 Key Point Information

[0245]

[0246] S1805, when the preprocessed image includes multiple maps of province A, determine the target map corresponding to the target detection area.

[0247] Electronic devices in Figure 20 After key point recognition is performed on the preprocessed image shown in (a), see [link to image]. Figure 20 As shown in (b), key points in two regions A can be detected separately (i.e., and ), the key points of two regions B (i.e. and ), the key points of the two regions C (i.e. and ), and a key point in region D (i.e. However, electronic devices cannot determine key points. , , , , , and Which key points belong to the land? Figure 1 Which key points belong to the land? Figure 2 Therefore, electronic devices need to divide the key points into sections and determine which key points belong to the same map.

[0248] In one possible implementation, the electronic device can determine key points on the same map based on the distance and positional relationship between various target detection areas.

[0249] For the key points of regions A and B, firstly, see Figure 21 As shown in (a1), the electronic device can perform full connectivity on key points in regions A and B, i.e., connect... and , and , and , and Subsequently, regarding the map of province A, since region B is located in the lower left corner of region A, and and The positional relationship between them is not satisfied. Therefore, see [link / reference needed]. Figure 21 As shown in (a2), the electronic device is disconnected. and The connections between them. Finally, the electronic device determines... and Distance between , and Distance between , and distance And determine the minimum distance among them. The minimum distance is... For example, then we can determine and They are located on the same map. It is impossible for key points in other areas B to be located on the same map. Therefore, see Figure 21 As shown in (a3), the electronic device is disconnected. and Connect the points. In the remaining connections, the key points are located on the same map. and Located on the same map, and Located on the same map.

[0250] Referring to the previous description of the key point division of region A and region B, and combining it with... Figure 21 (b1) to Figure 21 As shown in (b3), the electronic device can determine and Belonging to the same map, and They belong to the same map. Additionally, there is only one key point in region D. ,and and The distance is closest, therefore, determine and They belong to the same map.

[0251] In summary, see Figure 21 As shown in (c), the electronic device can determine , , , They belong to the same map, and , and They belong to the same map.

[0252] It should be understood that if the preprocessed image contains only one target map (e.g., a map of province A), then step S1805 is not required.

[0253] S1806 If the target detection area is not identified, the electronic device determines that there is a first violation problem in the target detection area.

[0254] For regions A and B, the common issue is misdrawn boundary lines. Therefore, regions A and B are usually recognizable by electronic devices and do not present a primary violation. However, in some embodiments, if the electronic device fails to recognize boundary regions such as region A or region B, it can be determined that there is a missing boundary line drawing issue. This missing boundary line drawing issue is also considered a primary violation.

[0255] For region C, since region C is an island, if the electronic device does not recognize region C in the preprocessed image, then region C is determined to have the island omission problem in the first violation problem.

[0256] For region D, since region D is a region illustration, if the electronic device does not recognize region D in the preprocessed image, it is determined that region D has the problem of missing region illustration in the first violation problem.

[0257] For example, for Figure 19 In the preprocessed image shown in (b), because the preprocessed image includes two maps of Province A, the electronic device did not recognize the location. Figure 1 Therefore, it can be determined that region D is... Figure 1 Region D has a missing illustration issue, which is a first violation issue.

[0258] S1807 If a target detection area is identified, the electronic device determines the corresponding region image from the preprocessed image based on the key point information of the target detection area.

[0259] In this embodiment, according to the detection task requirements, the electronic device needs to determine the regional images of regions A and B to detect whether they have the boundary misdrawing problem in the second violation problem; it also needs to determine the regional image of region C to detect whether it has the local area color difference problem in the second violation problem. Region D usually does not have the second violation problem, therefore, it is not necessary to determine the regional image of region D.

[0260] The method for determining the region images of regions A, B, and C is the same. In one possible implementation, the electronic device can determine the size of the region image of the target detection region based on the average distance L between keypoints of at least two target detection regions, and then combine this with the position of the keypoints to determine the region image. This is illustrated below.

[0261] In a map of province A, taking regions A, B, and C as examples, the average distance between keypoints corresponding to each target detection region is... .in, Let be the distance between key points in region A and key points in region B of the image to be detected. Let be the distance between key points in region B and key points in region C of the image to be detected. Let be the distance between key points in region A and key points in region C of the image to be detected.

[0262] For each region of each target map, the electronic device has a preset zoom factor. ,in, This represents the scaling factor for the horizontal edge of the region image. This represents the scaling factor for the vertical edge of the region image. The region image for each region can be calculated using the following formula.

[0263]

[0264] in, The length of the horizontal side of the region image. Let L be the length of the vertical side of the region image, and L be the average distance between keypoints corresponding to each target detection region in the image to be detected. This is the scaling factor for the horizontal side of the region image. This is the scaling factor for the vertical edge of the region image.

[0265] In this embodiment, the size of the region image can be represented by [a, b]. For example, [10, 15] can represent that the length of the horizontal side of the region image is 10 pixels and the length of the vertical side is 15 pixels.

[0266] However, the image to be detected provided in this embodiment contains two maps of Province A, and these two maps of Province A are different in size. Therefore, the electronic device needs to determine the size of its region image based on each map of Province A.

[0267] For the land Figure 1 Electronic devices according to , , average distance between and the presets for each area That is, can be determined Figure 1 The dimensions of the region images for each target detection area. For the ground Figure 2 Electronic devices based on key points , , average distance between and the presets for each area That is, can be determined Figure 2 The size of the region image for each target detection area.

[0268] against Figure 20 The preprocessed image shown in (b) can be determined using the method described above. , Based on this, if the aspect ratios of the preset region images for regions A, B, and C are [1 / 1.5, 1 / 1.9], [1 / 1.5, 1 / 1.2], and [1 / 1.625, 1 / 2], respectively, then in the... Figure 1 In the image, the dimensions of regions A, B, and C are [67, 51], [67, 81], and [60, 49], respectively. Figure 2 In the image, the dimensions of the region images of region A, region B and region C are [84, 69], [84, 106] and [78, 56], respectively.

[0269] In summary, electronic devices can obtain, for example, the regional image information shown in Table 5, which includes the map where each target detection region is located, the name of the target detection region, the name of the key point, the coordinates of the center position of the regional image (i.e., the coordinates of the key point), and the correspondence between the regional image size and the target detection region.

[0270] Table 5. Regional Image Information

[0271]

[0272] Based on the correspondence in Table 5, the electronic device can determine the region image of each target detection area in the preprocessed image. For example, for... Figure 19 The electronic device can determine the image to be detected shown in (a) above. Figure 1 The region images of regions A, B, and C are respectively Figure 22 (a1) Figure 22 (b1) and Figure 22 (c1) in the middle. And the ground Figure 2 The region images of regions A, B, and C are respectively Figure 22 (a2) Figure 22 (b2) and Figure 22 (c2) in the middle.

[0273] S1808, the electronic device determines whether a second violation exists in the corresponding target detection area based on the area image.

[0274] In this embodiment, depending on the type of target detection area, the electronic device can refer to Table 1 to use a boundary recognition model to identify whether area A and area B have boundary misdrawing issues in the second violation problem. A color recognition model is used to identify whether area C has local color discrepancies in the second violation problem.

[0275] For example, for Figure 22 The location shown in (a1) Figure 1 The region image of region A in the middle, Figure 22 The location shown in (b1) Figure 1 The region image of region B in the middle. Figure 22 The location shown in (a2) Figure 2 The region image of region A in the middle and Figure 22 The location shown in (b2) Figure 2 In the regional image of region B, the electronic device's violation classification scores, determined by the boundary recognition model, are 0.966, 0.977, 0.960, and 0.298, respectively. Therefore, the violation classification score for this region is determined to be... Figure 1 Region A, Region B, and the land Figure 2 Region A in the document is compliant. And the land... Figure 2Region B in the document contains a boundary misdrawing issue, which is a second violation.

[0276] For example, for Figure 22 The land described in (c1) Figure 1 In the image of region C, the electronic device, through a color recognition model, can determine that its violation classification score is 0.841, which is greater than the threshold of 0.5. Therefore, it is determined that... Figure 1 Region C in the model does not exhibit the problem of localized color variations. However, for... Figure 22 The location shown in (c2) Figure 2 In the image of region C, the electronic device, through a color recognition model, can determine its violation classification score as 0.216, which is less than the threshold of 0.5. Therefore, it is determined that... Figure 2 Region C in the image exhibits a problem of localized color variations.

[0277] Furthermore, electronic devices can also analyze and detect preprocessed images using regression recognition models to determine the size of the region inset in region D. In one example, for... Figure 19 The preprocessed image shown in (b) can be determined to have a region inset size of [53, 79].

[0278] Based on the above steps S1801-S1808, the electronic device completes the process of... Figure 19 After recognizing the image to be detected shown in (a), a detection result is generated. For example, the detection result is shown below:

[0279] {

[0280] "success_code":200;

[0281] "error_message":"success";

[0282] "suggestion":"abnormal"; #Final review result

[0283] “abnormal area”:[

[0284] "B_area", "C_area" #Violation ]

[0286] “map_classify_results”:[ #Target map classification score]

[0287] {

[0288] “label”:“otners”,

[0289] "confidence": 0.0;

[0290] }

[0291] {

[0292] "label": "map",

[0293] "confidence": 1.0;

[0294] }

[0295] ,

[0296] "regional_location":

[0298] "regional_location": 1, #地 Figure 1

[0299] {

[0300] "label": "A_area", #区域类别

[0301] "regional confidence": 0.943, #关键点响应值

[0302] "location": [[ #关键点坐标

[0303] 256, 24

[0305] ,

[0306] "suggestion": "normal", #违规分类结果判断

[0307] "type": 0, #违规类型

[0308] "normal_score": 0.966 #违规分类得分

[0309] },

[0310] {

[0311] "label": "B_area", #区域类别

[0312] "regional_confidence": 0.955, #关键点响应值

[0313] "location": [[ #关键点坐标

[0314] 116, 114 ​

[0316] ],

[0317] "suggestion": "abnormal", #Judgment of violation classification results

[0318] "type":0, #Type of violation

[0319] “normal_score”: 0.977 # Score for violation category

[0320] },

[0321] {

[0322] “label”:“C_area”, #area category

[0323] “regional_confidence”: 0.963, #Key point response value

[0324] “location”:[#key point coordinates]

[0325] 320, 146

[0327] ],

[0328] "suggestion": "abnormal", #Judgment of violation classification results

[0329] "type":0, #Type of violation

[0330] “normal_score”: 0.841 # Score for violation category

[0331] },

[0332] {

[0333] “label”:“D_area”, #area category

[0334] “regional_confidence”: 0.909, #Key point response value

[0335] “location”:[#key point coordinates]

[0336] 396, 157

[0338] ],

[0339] “bounding_box”:[

[0340] 396,

[0341] 157,

[0342] 53, 79

[0345] “suggestion”:“normal”, # Classification result judgment of violations

[0346] “type”:0, # Violation type

[0347] }

[0348] ,

[0350] “regional_location”:2, # Region Figure 2

[0351] {

[0352] “label”:“A_area”, # Region category

[0353] “regional confidence”:0.956, # Key point response value

[0354] “location”:[ # Key point coordinates

[0355] 250, 208

[0357] ,

[0358] “suggestion”:“normal”, # Classification result judgment of violations

[0359] “type”:0, # Violation type

[0360] “normal_score”:0.960 # Classification score of violations

[0361] },

[0362] {

[0363] “label”:“B_area”, # Region category

[0364] “regional_confidence”:0.973, # Key point response value

[0365] “location”:[ # Key point coordinates

[0366] 78, 396

[0368] , ​​

[0369] "suggestion": "normal", # Judgment of violation classification results

[0370] "type":1, #Type of violation

[0371] “normal_score”: 0.298 # Score for violation category

[0372] },

[0373] {

[0374] “label”:“C_area”, #area category

[0375] “regional_confidence”: 0.875, #Key point response value

[0376] “location”:[ #keypoint coordinates]

[0377] 378, 424

[0379] ],

[0380] "suggestion": "normal", # Judgment of violation classification results

[0381] "type":3, #Type of violation

[0382] “normal_score”: 0.216 # Score for violation category

[0383] } ]

[0385] ],

[0386] "version": 1.0

[0387] }

[0388] It should be noted that, in this embodiment, if the electronic map identifies a violation in at least one target detection area on the image to be detected, the map is determined to be a problematic map, and the detection result is displayed on the map. This detection result is used to indicate that the map is a problematic map. For example, regarding... Figure 19 The detection result information for the image to be detected shown in (a) can be found in [reference]. Figure 23 As shown, the detected areas A, B, C, and D are highlighted on the target map, along with the text message "Compliant Map". Figure 1 "Region A: Compliant, Region B: Compliant, Region C: Compliant, Region D: Compliant" and "Problem Map" Figure 2Region A: Misdrawn boundary; Region B: Misdrawn boundary; Region C: Dissimilar color in a local area; Region D: Omitted illustration in the region.

[0389] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0390] This application also provides an electronic device, see [link to relevant documentation] Figure 24 The electronic device includes a memory 2401, a processor 2402, and a computer program 2403 stored in the memory 2401 and executable on the processor 2402. When the processor 2402 executes the computer program, it implements the problem map detection method provided in this embodiment.

[0391] This application also provides a problem map detection system, see [link to relevant documentation]. Figure 25 As shown, the system includes a client and a server, and is configured to perform the following steps.

[0392] S2501, the client obtains the image to be detected and the detection strategy.

[0393] In this embodiment, the client can be a desktop computer, mobile phone, tablet computer, wearable device, in-vehicle device, AR / VR device, laptop computer, UMPC, netbook, PDA, or other terminal device. The main data sources of the images to be detected collected by the client are individual users, enterprise users, or content providers (CPs).

[0394] Additionally, see optional, see Figure 26 As shown, users can select different target maps in the client, for example... Figure 26 In the process, the target map is selected as the map of province A. In response to the user's selection, the client can generate a detection strategy according to the user's needs. This detection strategy includes the identification information of the target map, which is used by the server to determine the target map for this detection during the process of detecting problematic maps.

[0395] In other embodiments, if the user does not set a target map, the client can also generate a detection strategy based on default requirements, such as including identification information of certain default countries or regions, for the server to determine the target map for this detection during the process of detecting problematic maps.

[0396] S2502, the client sends a problem map detection request to the server, which carries the image to be detected and the detection strategy.

[0397] This issue map detection request is used to ask the server to identify whether the image to be detected contains an issue map and to determine the type of issue. To facilitate the transmission of the image over the internet, the client can convert the image to base64 format before sending it to the server, thus improving data transmission quality. The detection strategy may include the target map's identification information, which the server uses to determine the target map during the issue map detection process.

[0398] Alternatively, the client can preprocess the image to be detected locally and send the preprocessed image to the server via a map detection request. The preprocessed image is usually smaller than the original image to be detected, making it easier to transmit to the server.

[0399] S2503, the server performs problem map detection on the image to be detected according to the detection strategy and obtains the detection results.

[0400] After receiving a map detection request, the server needs to decode the image data carried in the request and perform parameter validation. Parameter validation includes verifying whether the format of the image to be detected is a format supported by the server, such as JPG.

[0401] In this embodiment, the specific content of S2503 can be found in the problem map detection methods shown in S301-S306, S701-S709, or S1801-S1808 above, which perform problem map detection on the image to be detected and obtain the detection result. This embodiment will not repeat the details here.

[0402] S2504, the server sends the test results to the client.

[0403] In some embodiments, the client does not have the capability to identify problem maps locally. In this case, the problem map detection system provided in this application allows the electronic device to send the locally collected image to be detected to the server for problem map identification, resulting in a better user experience. Furthermore, since the server has stronger computing power, the recognition efficiency is higher compared to performing image recognition locally on the client.

[0404] In addition, in some other embodiments, the client in the problem map detection system may not have a detection strategy, but instead send the image to be detected to the server, so that the server can perform problem map detection on the image to be detected according to a preset detection strategy, obtain the detection result, and send the detection result to the client.

[0405] This embodiment provides a computer program product, which includes a program that, when run by an electronic device, enables the electronic device to perform the problem map detection method shown in the above embodiments.

[0406] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the problem map detection method shown in the above embodiments.

[0407] This application provides a chip including a memory and a processor. The processor executes a computer program stored in the memory to control the electronic device to perform the problem map detection method shown in the above embodiments.

[0408] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0409] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0410] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0411] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0412] In the above context, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0413] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0414] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting problematic maps, characterized in that, The method includes: Acquire the image to be detected; Identify all target maps in the image to be detected, each target map including at least one target detection region; Each target map is detected using a problem map detection model to determine the violation type of each target detection area in each target map; The violation type includes at least one of the following types: no violation, first violation issue, or second violation issue; The phrase "not in violation" indicates that the corresponding target detection area has been identified and that the drawing of the target detection area is compliant. The first violation issue indicates that the corresponding target detection area was not identified; The second violation is used to indicate that the corresponding target detection area has been identified, but the target detection area has been drawn incorrectly; The step of detecting each target map using a problem map detection model to determine the violation type of each target detection region in each target map includes: The response value of the target detection area is determined by the key point detection model in the problem map detection model. When the target detection area is a boundary area, and the response value of the key point in the boundary area is less than the threshold, it is determined that there is a problem of missing boundary area drawing. When the target detection area is an island, and the response value of the key point of the island is less than the threshold, it is determined that the island has a problem of missing drawing; When the target detection area is a region illustration, and the response value of the key point of the region illustration is less than a threshold, it is determined that the region illustration has a problem of missing drawing.

2. The method according to claim 1, characterized in that, Each target map is detected using a problem map detection model to determine the violation type of each target detection region in each target map, including: The image to be detected is detected using the semantic segmentation network in the problem map detection model. Based on the detection results, determine the violation type for each target detection area in each target map.

3. The method according to claim 2, characterized in that, When the target detection region is a boundary region, the semantic segmentation network in the problem map detection model is used to detect the image to be detected; the violation type of each target detection region in each target map is determined based on the detection results, specifically including: The semantic segmentation network is used to identify the first map mask where the boundary region is located; The first map mask is matched with the preset compliant map to determine the contour matching relationship; Based on the contour matching relationship and the coordinates of the boundary region in the compliant map, determine the key point coordinates of the boundary region in the first map mask; Based on the coordinates of the key points, a region image of the boundary area is extracted from the image to be detected; The boundary recognition model is used to identify the region image of the boundary area in order to determine whether there is a problem of misdrawn boundary in the boundary area.

4. The method according to claim 2, characterized in that, When the target detection region is an island, the semantic segmentation network in the problem map detection model is used to detect the image to be detected; the violation type of each target detection region in each target map is determined based on the detection results, specifically including: The semantic segmentation network is used to identify the second map mask where the island is located; If the second map mask is detected, it is determined that there is no problem of missing islands in the map. If the second map mask is not detected, it is determined that there is a problem of missing islands in the map.

5. The method according to claim 4, characterized in that, When the target detection area is an island, the image to be detected is detected by the semantic segmentation network in the problem map detection model. Determining the violation type for each target detection area in each target map based on the detection results also includes: The coordinates of key points on the island are determined based on the second map mask; Based on the coordinates of the key points of the island, a region image of the island is extracted from the image to be detected; By using a color recognition model, the regional image of the island is compared with the target map in the image to be detected to determine whether there is a problem of local color difference in the island.

6. The method according to claim 2, characterized in that, When the target detection region is a region inset, the image to be detected is detected using the semantic segmentation network in the problem map detection model; the violation type of each target detection region in each target map is determined based on the detection results, specifically including: The semantic segmentation network is used to identify the third map mask where the region inset in the image to be detected is located; If the third map mask is identified, it is determined that there is no problem of missing area illustrations in the area illustration; If the third map mask is not identified, it is determined that there is a problem with the omission of area illustrations.

7. The method according to claim 1, characterized in that, When the image to be detected includes multiple target maps, the method further includes: The key point detection model is used to identify key points in all target detection areas of the multiple target maps. All key points are classified according to their positional relationships to determine which key points belong to the same target map. Based on key points belonging to the same target map, determine the target detection area belonging to the same target map; For each target map, determine the violation type of the corresponding target detection area.

8. The method according to claim 1, characterized in that, When the target detection region is a boundary region, the violation type for each target map is determined by using the problem map detection model to detect each target map and identify the violation type for each target detection region in each target map. When the response value of the boundary region is greater than or equal to the threshold, the coordinates of the key points in the boundary region are determined by the key point detection model. Based on the coordinates of the key points, a region image of the boundary area is extracted from the image to be detected; The boundary recognition model is used to identify the region image of the boundary area in order to determine whether there is a problem of misdrawn boundary in the boundary area.

9. The method according to claim 1, characterized in that, When the target detection area is an island, the problem map detection model is used to detect each of the target maps, and the violation type of each target detection area in each target map is determined by further steps: When the response value of the key point of the island is greater than or equal to the threshold, the coordinates of the key point of the island are determined by the key point detection model. Based on the key point coordinates of the island, a region image of the island is extracted from the image to be detected; By using a color recognition model, the regional image of the island is compared with the target map in the image to be detected to determine whether there is a problem of local color difference in the island.

10. The method according to claim 3, 5, 8 or 9, characterized in that, Based on the key point coordinates of the target detection region, the corresponding region image is extracted, including: The pixel area of ​​the region image is determined based on the pixel area of ​​the image to be detected. The width and height of the target detection region are determined based on the preset aspect ratio of the target detection region and the pixel area of ​​the region image; Based on the key point coordinates of the target detection region, as well as the width and height of the target detection region, a region image of the target detection region is extracted from the image to be detected.

11. The method according to claim 3, 5, 8 or 9, characterized in that, Based on the key point coordinates of the target detection region, the corresponding region image is extracted, including: Determine the average distance between key points in multiple target detection areas on the same target map; The width and height of the target detection area are determined based on the preset aspect ratio of the target detection area and the average distance. Based on the key point coordinates of the target detection region, as well as the width and height of the target detection region, a region image of the target detection region is extracted from the image to be detected.

12. A problem map detection system, characterized in that, Includes both client and server; The client is configured as follows: Send the image to be detected to the server; The server is configured as follows: Identify all target maps in the image to be detected, each target map including at least one target detection region; Each target map is detected using a problem map detection model, and detection results are obtained. The detection results include the violation type of each target detection area in each target map. Send the detection result to the client; The violation type includes at least one of the following types: no violation, first violation issue, or second violation issue; The phrase "not in violation" indicates that the corresponding target detection area has been identified and that the drawing of the target detection area is compliant. The first violation issue indicates that the corresponding target detection area was not identified; The second violation is used to indicate that the corresponding target detection area has been identified, but the target detection area has been drawn incorrectly; The step of detecting each target map using a problem map detection model to determine the violation type of each target detection region in each target map includes: The response value of the target detection area is determined by the key point detection model in the problem map detection model. When the target detection area is a boundary area, and the response value of the key point in the boundary area is less than the threshold, it is determined that there is a problem of missing boundary area drawing. When the target detection area is an island, and the response value of the key point of the island is less than the threshold, it is determined that the island has a problem of missing drawing; When the target detection area is a region illustration, and the response value of the key point of the region illustration is less than a threshold, it is determined that the region illustration has a problem of missing drawing.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the problem map detection method as described in any one of claims 1-11.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the problem map detection method as described in any one of claims 1-11.

15. A computer program product, the program product comprising a program, characterized in that, When the program is run by an electronic device, the electronic device enables the problem map detection method as described in any one of claims 1-11.

Images