Protection wallpaper type identification method and device, electronic equipment and storage medium

Abstract

The invention relates to the field of wallpaper detection, in particular to a protection wallpaper type identification method and device, electronic equipment and a storage medium. The method comprises the steps that a target image is detected and recognized through a pre-trained target detection model, a first detection frame, a second detection frame and a third detection frame are obtained, the first detection frame is used for marking a corresponding area of a door in the target image, and the second detection frame is used for marking a corresponding area of a window in the target image; the third detection frame is used for marking a corresponding area of the protection wallpaper in the target image; based on the relative position relation of the first detection frame, the second detection frame and the third detection frames, the type of the protection wallpaper corresponding to each third detection frame is determined, and the type of the protection wallpaper comprises protection wallpaper of a door, standard wallpaper of a window or protection wallpaper of other types.

Description

Technical Field

[0001] The present disclosure relates to the field of wallpaper detection, and more specifically, to a method, device, electronic device and storage medium for identifying the type of protective wallpaper. Background Art

[0002] In home decoration scenarios, doors and windows are often covered with protective wallpaper to avoid damage during the decoration process. However, existing detection technologies cannot effectively distinguish between door and window protective wallpaper, especially when windows and doors look similar. For example, when detecting protective wallpaper, the existing YOLOv5-based object detection model will mistakenly identify the protective wallpaper on some windows as door protective wallpaper. The main reason for this misjudgment is that windows and doors have certain similarities in appearance. Summary of the invention

[0003] One purpose of the present disclosure is to provide a method for identifying the type of protective wallpaper, so as to solve the problem that it is difficult to distinguish between paging protective wallpaper and door protective wallpaper.

[0004] According to a first aspect of the present disclosure, a method for identifying a type of protective wallpaper is provided, comprising:

[0005] Detect and identify the target image through a pre-trained target detection model to obtain a first detection frame, a second detection frame, and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image;

[0006] Based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, and the type of protective wallpaper includes protective wallpaper for doors, standard wallpaper for windows, or other types of protective wallpaper.

[0007] Optionally, the determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame includes:

[0008] Determine a first intersection-over-union ratio between each of the third detection frames and each of the first detection frames;

[0009] Determine a second intersection-over-union ratio between each of the third detection frames and each of the second detection frames;

[0010] Based on the first intersection-over-union ratio and the second intersection-over-union ratio, a type of protective wallpaper corresponding to each third detection frame is determined.

[0011] Optionally, determining the type of protective wallpaper corresponding to each third detection frame based on the first intersection-over-union ratio and the second intersection-over-union ratio includes:

[0012] For any of the third detection frames, when the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for a door;

[0013] For any of the third detection frames, when the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows;

[0014] For any of the third detection frames, when the first intersection-and-union ratio is less than or equal to a first threshold and the second intersection-and-union ratio is less than or equal to a second threshold, it is determined that the corresponding protection wallpaper is another type of protection wallpaper.

[0015] Optionally, the third detection frame includes a fourth detection frame and a fifth detection frame, the fourth detection frame is used to mark the corresponding area of ​​the door's protective wallpaper in the target image, and the fifth detection frame is used to mark the corresponding area of ​​the window's protective wallpaper in the target image.

[0016] Optionally, the determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame includes:

[0017] Determine a first intersection-over-union ratio between each of the fourth detection frames and each of the first detection frames;

[0018] For any of the fourth detection frames, when the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for a door;

[0019] When the first intersection-and-union ratio is less than or equal to a first threshold, determining a second intersection-and-union ratio between the fourth detection frame and each of the second detection frames;

[0020] When the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows;

[0021] When the second intersection-over-union ratio is less than or equal to a second threshold, it is determined that the type of the corresponding protection wallpaper is other types of protection wallpaper.

[0022] Optionally, the determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame further includes:

[0023] Determine a second intersection-over-union ratio between each of the fifth detection frames and each of the second detection frames;

[0024] For any of the fifth detection frames, when the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows;

[0025] When the second intersection-and-union ratio is less than or equal to a first threshold, determining a first intersection-and-union ratio between the fifth detection frame and each of the first detection frames;

[0026] When the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protection wallpaper is a door protection wallpaper;

[0027] When the first intersection-over-union ratio is less than or equal to a first threshold, it is determined that the type of the corresponding protection wallpaper is other types of protection wallpaper.

[0028] Optionally, before detecting and identifying the target image by using a pre-trained target detection model, the method further includes:

[0029] Acquire training data and annotate the training data, wherein the annotated types include doors, windows, door protection wallpapers and window protection wallpapers, and the training data at least includes window images under a strong light environment, and the window images under a strong light environment are extracted based on a SAM image segmentation algorithm;

[0030] The target detection model is trained based on the training data to obtain the pre-trained target detection model.

[0031] According to a second aspect of the present disclosure, there is provided a device for identifying the type of protective wallpaper, comprising:

[0032] a recognition module, configured to detect and recognize the target image through a pre-trained target detection model, and obtain a first detection frame, a second detection frame, and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image;

[0033] A determination module is used to determine the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, and the type of protective wallpaper includes protective wallpaper for doors, standard wallpaper for windows, or other types of protective wallpaper.

[0034] According to a third aspect of the present disclosure, an electronic device is provided, including a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of any one of the methods described in the first aspect are implemented.

[0035] According to a fourth aspect of the present disclosure, there is provided a storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0036] One technical effect of the present disclosure is that a method for identifying the type of protective wallpaper is provided, which can obtain the detection frames of doors, windows and protective wallpaper in the target image through a pre-trained target detection model, and based on the relative position relationship of the detection frames, further determine whether the protective wallpaper is the protective wallpaper of the door, window or other object, thereby solving the problem that the existing detection technology cannot effectively distinguish between the protective wallpaper of the door and the protective wallpaper of the window.

[0037] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0039] Figure 1 is a flow chart of a method for identifying a type of protective wallpaper according to one embodiment;

[0040] Figure 2 is a schematic diagram of a structure of a device for identifying the type of protective wallpaper according to an embodiment

[0041] Figure 3 is a schematic structural diagram of an electronic device according to an embodiment; DETAILED DESCRIPTION

[0042] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0043] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0044] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0045] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0046] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0047] The present application embodiment discloses a method for identifying the type of protected wallpaper, such as Figure 1 As shown, it includes step S11 to step S12.

[0048] Step S11, detect and identify the target image through the pre-trained target detection model to obtain a first detection frame, a second detection frame and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image.

[0049] In an example, the target detection model can be a target detection model such as YOLO, SSD, R-CNN, etc. The target detection model is a multi-task target detection model, and training data can be acquired in advance to train it to recognize various objects in the image, such as doors, windows, protective wallpaper, etc.

[0050] In the first example of this embodiment, before detecting and identifying the target image through a pre-trained target detection model, the method also includes: obtaining training data and annotating the training data, wherein the annotated types include doors, windows, door protective wallpapers, and window protective wallpapers, and the training data at least includes window images under strong lighting environments, and the window images under strong lighting environments are extracted based on the SAM image segmentation algorithm; training the target detection model based on the training data to obtain a pre-trained target detection model.

[0051] In this embodiment, a plurality of surveillance images of home decoration scenes or construction site scenes may be acquired in advance, and then the doors, windows and protective wallpapers in the training data may be labeled using a labeling tool. In one example, the labeling of the protective wallpaper may not be marked with a specific type, and correspondingly, the target detection model trained using the training data will not identify the specific type of protective wallpaper during recognition. In another example, the labeling of the protective wallpaper may be marked with a specific type, such as protective wallpaper for doors, protective wallpaper for windows, and so on. The corresponding trained target detection model may also further detect the protective wallpaper for doors and protective wallpaper for windows. After the training data is acquired and labeled, the target detection model may be trained based on the training data.

[0052] In this embodiment, in a strong light environment, the detection effect of windows is often poor and easily affected by exposure. To address this problem, the present application can segment the window image in a strong light environment based on SAM (Segment Anything Model), generate a two-dimensional mask to represent the window area in a strong light environment, and input it into the target detection model for training, so as to further improve the detection accuracy of the model for windows and window protection wallpapers under complex lighting conditions.

[0053] In this embodiment, after the target image is input into the pre-trained target detection model, the target detection model can recognize the target image and identify the door, window and protective wallpaper in the target image. In this embodiment, the target detection model can output multiple detection frames, including a first detection frame marking the area where the door is located, a second detection frame marking the area where the window is located and a third detection frame marking the area corresponding to the protective wallpaper. In this embodiment, the number of the first detection frame, the second detection frame and the third detection frame can be 0, 1 or more.

[0054] Step S12, based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame, determining the type of protective wallpaper corresponding to each third detection frame, the types of protective wallpaper including protective wallpaper for doors, standard wallpaper for windows or other types of protective wallpaper.

[0055] In this embodiment, after the detection frames of the door, window and protective wallpaper in the target image are determined, the type of each protective wallpaper can be determined based on the positional relationship between these detection frames. For example, the coordinates of the center point of each detection frame can be determined, and then for each third detection frame, the distance between the third detection frame and the center coordinates of each other first detection frame or second detection frame will be determined. When the distance to the center point of the first detection frame is less than a threshold, the wallpaper type of the third detection frame can be determined to be the protective wallpaper of the door. When the distance to the center point of the second detection frame is less than a threshold, the wallpaper type of the third detection frame can be determined to be the protective wallpaper of the window. When the distance to any detection frame is not less than a threshold, the detection wallpaper can be determined to be other types of detection wallpaper, such as floor protective wallpaper.

[0056] In this example, a method for identifying the type of protective wallpaper is provided. Through a pre-trained target detection model, the detection frames of doors, windows and protective wallpapers can be obtained in the target image. Based on the relative position relationship of the detection frames, it can be further determined whether the protective wallpaper is the protective wallpaper of doors, windows or other objects. This solves the problem that the existing detection technology cannot effectively distinguish between the protective wallpaper of doors and the protective wallpaper of windows, realizes construction specification detection for home decoration construction scenarios, and effectively avoids the problem that the construction party fails to post protective wallpapers for doors and windows according to construction requirements.

[0057] In an example of the present embodiment, based on the relative position relationship of the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, including: determining a first intersection-and-union ratio of each third detection frame and each first detection frame; determining a second intersection-and-union ratio of each third detection frame and each second detection frame; based on the first intersection-and-union ratio and the second intersection-and-union ratio, determining the type of protective wallpaper corresponding to each third detection frame.

[0058] The relative position relationship between the first detection box, the second detection box and the third detection box can be expressed in the form of intersection over union (IoU), which is calculated by the IoU calculation formula:

[0059]

[0060] Wherein, A represents the third detection frame, and B represents the first detection frame or the second detection frame. In this example, for each third detection frame, its intersection and union ratio with each first detection frame and the second detection frame can be calculated respectively, and the overlap between the third detection frame and the other detection frames can be determined through the intersection and union ratio, so as to determine which door or window the wallpaper corresponding to the third detection frame overlaps with, so as to determine the type of protective wallpaper.

[0061] In an example of the present embodiment, based on the first intersection-and-union ratio and the second intersection-and-union ratio, the type of protective wallpaper corresponding to each third detection frame is determined, including: for any third detection frame, when the first intersection-and-union ratio is greater than a first threshold, the type of the corresponding protective wallpaper is determined to be the protective wallpaper for the door; for any third detection frame, when the second intersection-and-union ratio is greater than a second threshold, the type of the corresponding protective wallpaper is determined to be the protective wallpaper for the window; for any third detection frame, when the first intersection-and-union ratio is less than or equal to the first threshold and the second intersection-and-union ratio is less than or equal to the second threshold, the corresponding protective wallpaper is determined to be other types of protective wallpaper.

[0062] In this embodiment, the intersection-and-union ratio reflects the overlap between the two detection frames. The closer it is to 1, the closer they are to overlap, and the closer it is to 0, the less they overlap. Therefore, the values ​​of the first threshold and the second threshold can be set according to actual needs, and the first threshold and the second threshold can be the same or different. When the intersection-and-union ratio of a third detection frame and a first detection frame is greater than the threshold, that is, when the first intersection-and-union ratio is greater than the first threshold, it means that the third detection frame overlaps more with the door detection frame, that is, the protective wallpaper corresponding to the detection frame is the door protection wallpaper. When the intersection-and-union ratio of a third detection frame and a second detection frame is greater than the threshold, that is, when the second intersection-and-union ratio is greater than the second threshold, it means that the third detection frame overlaps more with the window detection frame, that is, the protective wallpaper corresponding to the detection frame is the window protection wallpaper. When the intersection-and-union ratio of a third detection frame and each first detection frame and second detection frame is less than the threshold, it means that the third detection frame does not have a relatively overlapping detection frame, that is, the protective wallpaper corresponding to the third detection frame is other types of protective wallpaper.

[0063] In an example of this embodiment, the third detection frame includes a fourth detection frame and a fifth detection frame, the fourth detection frame is used to mark the corresponding area of ​​the door's protective wallpaper in the target image, and the fifth detection frame is used to mark the corresponding area of ​​the window's protective wallpaper in the target image.

[0064] During training, the pre-trained target detection model can perform more detailed training on protective wallpapers, and can use the model to identify window protective wallpapers and door protective wallpapers. That is to say, the detection frame of the protective wallpaper in the target image recognition result also includes two detection frames corresponding to the window protective wallpapers and the door protective wallpapers.

[0065] In an example of the present embodiment, based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, including: determining a first intersection-and-union ratio of each fourth detection frame with each first detection frame; for any fourth detection frame, when the first intersection-and-union ratio is greater than a first threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for the door; when the first intersection-and-union ratio is less than or equal to the first threshold, determining the second intersection-and-union ratio of the fourth detection frame with each second detection frame; when the second intersection-and-union ratio is greater than the second threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for the window; when the second intersection-and-union ratio is less than or equal to the second threshold, determining the type of the corresponding protective wallpaper to be other types of protective wallpaper.

[0066] In this example, for the fourth detection frame obtained by the target detection model identifying the door's protective wallpaper, the intersection and union ratio between each fourth detection frame and the first detection frame corresponding to the door area can be determined first to verify whether the model detection result is correct. When the intersection and union ratio of a fourth detection frame and a first detection frame is greater than the threshold, that is, when the first intersection and union ratio is greater than the first threshold, it means that the protective wallpaper corresponding to the detection frame is the door's protective wallpaper, and the detection model recognition result is correct. When the intersection and union ratio of a fourth detection frame and a first detection frame is less than or equal to the threshold, that is, when the first intersection and union ratio is less than or equal to the first threshold, it means that the protective wallpaper corresponding to the detection frame is not the door's protective wallpaper, and the detection model recognition result is inaccurate. At this time, it can be further determined whether the protective wallpaper is the window's protective wallpaper, and the second intersection and union ratio of the fourth detection frame and each second detection frame is determined. When the second intersection and union ratio is greater than the second threshold, it means that the protective wallpaper corresponding to the detection frame is the window's protective wallpaper. When the second intersection and union ratio is less than or equal to the second threshold, it means that the protective wallpaper is neither the door's protective wallpaper nor the window's protective wallpaper.

[0067] In an example of the present embodiment, based on the relative position relationship of the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, and it also includes: determining the second intersection-and-union ratio of each fifth detection frame and each second detection frame; for any fifth detection frame, when the second intersection-and-union ratio is greater than the second threshold, determining the type of the corresponding protective wallpaper as the protective wallpaper for the window; when the second intersection-and-union ratio is less than or equal to the first threshold, determining the first intersection-and-union ratio of the fifth detection frame and each first detection frame; when the first intersection-and-union ratio is greater than the first threshold, determining the type of the corresponding protective wallpaper as the protective wallpaper for the door; when the first intersection-and-union ratio is less than or equal to the first threshold, determining the type of the corresponding protective wallpaper as other types of protective wallpaper.

[0068] In this example, for the fifth detection frame obtained by the target detection model to identify the protective wallpaper of the window, the intersection and union ratio between each fifth detection frame and the second detection frame corresponding to the door area can be determined first to verify whether the model detection result is correct. When the intersection and union ratio of a certain fifth detection frame and a certain second detection frame is greater than the threshold, that is, when the second intersection and union ratio is greater than the second threshold, it means that the protective wallpaper corresponding to the detection frame is the protective wallpaper of the window, and the result of the detection model recognition is correct. When the intersection and union ratio of a certain fifth detection frame and a certain second detection frame is less than or equal to the threshold, that is, when the second intersection and union ratio is less than or equal to the second threshold, it means that the protective wallpaper corresponding to the detection frame is not the protective wallpaper of the window, and the result of the detection model recognition is inaccurate. At this time, it can be further determined whether the protective wallpaper is the protective wallpaper of the door, and the first intersection and union ratio of this fifth detection frame and each first detection frame is determined. When the first intersection and union ratio is greater than the second threshold, it means that the protective wallpaper corresponding to the detection frame is the protective wallpaper of the door. When the first intersection and union ratio is less than or equal to the first threshold, it means that the protective wallpaper is neither the protective wallpaper of the door nor the protective wallpaper of the window.

[0069] In this example, the model can be used to further identify the specific type of protective wallpaper, and then a secondary verification is performed based on the type. The intersection and union ratio of the protective wallpaper area of ​​the door or window identified by the model is first calculated with the corresponding door and window area to verify whether the model recognition result is accurate. If it is inaccurate, the intersection and union ratio with another area is calculated, which effectively reduces the amount of calculation and improves the recognition efficiency in images with more doors, windows, and wallpapers.

[0070] The present application also provides a device 100 for identifying the type of protective wallpaper. Figure 2 As shown, it includes: an identification module 101, which is used to detect and identify the target image through a pre-trained target detection model to obtain a first detection frame, a second detection frame and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image; a determination module 102, which is used to determine the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship of the first detection frame, the second detection frame and the third detection frame, and the type of protective wallpaper includes protective wallpaper for doors, standard wallpaper for windows or other types of protective wallpaper.

[0071] Optionally, based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, including: determining a first intersection-and-union ratio of each third detection frame and each first detection frame; determining a second intersection-and-union ratio of each third detection frame and each second detection frame; based on the first intersection-and-union ratio and the second intersection-and-union ratio, determining the type of protective wallpaper corresponding to each third detection frame.

[0072] Optionally, based on the first intersection-and-union ratio and the second intersection-and-union ratio, the type of protective wallpaper corresponding to each third detection frame is determined, including: for any third detection frame, when the first intersection-and-union ratio is greater than a first threshold, determining that the type of the corresponding protective wallpaper is the protective wallpaper for the door; for any third detection frame, when the second intersection-and-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is the protective wallpaper for the window; for any third detection frame, when the first intersection-and-union ratio is less than or equal to the first threshold and the second intersection-and-union ratio is less than or equal to the second threshold, determining that the corresponding protective wallpaper is other types of protective wallpaper.

[0073] Optionally, the third detection frame includes a fourth detection frame and a fifth detection frame, the fourth detection frame is used to mark the corresponding area of ​​the door's protective wallpaper in the target image, and the fifth detection frame is used to mark the corresponding area of ​​the window's protective wallpaper in the target image.

[0074] Optionally, based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, including: determining a first intersection-and-union ratio of each fourth detection frame and each first detection frame; for any fourth detection frame, when the first intersection-and-union ratio is greater than a first threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for the door; when the first intersection-and-union ratio is less than or equal to the first threshold, determining the second intersection-and-union ratio of the fourth detection frame and each second detection frame; when the second intersection-and-union ratio is greater than the second threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for the window; when the second intersection-and-union ratio is less than or equal to the second threshold, determining the type of the corresponding protective wallpaper to be other types of protective wallpaper.

[0075] Optionally, based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, determining the type of protective wallpaper corresponding to each third detection frame also includes: determining the second intersection-and-union ratio of each fifth detection frame and each second detection frame; for any fifth detection frame, when the second intersection-and-union ratio is greater than a second threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for windows; when the second intersection-and-union ratio is less than or equal to a first threshold, determining the first intersection-and-union ratio of the fifth detection frame and each first detection frame; when the first intersection-and-union ratio is greater than the first threshold, determining the type of the corresponding protective wallpaper to be protective wallpaper for doors; when the first intersection-and-union ratio is less than or equal to the first threshold, determining the type of the corresponding protective wallpaper to be other types of protective wallpaper.

[0076] Optionally, before detecting and identifying the target image through a pre-trained target detection model, the method also includes: obtaining training data and annotating the training data, wherein the annotated types include doors, windows, door protective wallpapers, and window protective wallpapers, and the training data at least includes window images under strong lighting environments, and the window images under strong lighting environments are extracted based on the SAM image segmentation algorithm; training the target detection model based on the training data to obtain a pre-trained target detection model.

[0077] like Figure 3 As shown, an embodiment of the present application further provides an electronic device 200, including a processor 201 and a memory 202, wherein the memory 202 stores computer instructions, and when the computer instructions are executed by the processor 201, the steps of any one of the methods in the embodiment of the method for identifying the type of protected wallpaper are implemented.

[0078] The embodiment of the present application also provides a storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, any one of the above-mentioned protection wallpaper type identification embodiments is implemented and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0079] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0080] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] The embodiments of the present disclosure may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure.

[0082] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0083] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0084] The computer program instructions for performing the operation of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the embodiments of the present disclosure.

[0085] Various aspects of the embodiments of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0088] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0089] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying the type of protective wallpaper, characterized in that: include: Detect and identify the target image through a pre-trained target detection model to obtain a first detection frame, a second detection frame, and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image; Based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame, the type of protective wallpaper corresponding to each third detection frame is determined, and the type of protective wallpaper includes protective wallpaper for doors, standard wallpaper for windows, or other types of protective wallpaper.

2. The method according to claim 1, characterized in that The determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame, and the third detection frame includes: Determine a first intersection-over-union ratio between each of the third detection frames and each of the first detection frames; Determine a second intersection-over-union ratio between each of the third detection frames and each of the second detection frames; Based on the first intersection-over-union ratio and the second intersection-over-union ratio, a type of protective wallpaper corresponding to each third detection frame is determined.

3. The method according to claim 1, characterized in that Determining the type of protective wallpaper corresponding to each third detection frame based on the first intersection-over-union ratio and the second intersection-over-union ratio includes: For any of the third detection frames, when the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for a door; For any of the third detection frames, when the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows; For any of the third detection frames, when the first intersection-and-union ratio is less than or equal to a first threshold and the second intersection-and-union ratio is less than or equal to a second threshold, it is determined that the corresponding protection wallpaper is another type of protection wallpaper.

4. The method according to claim 1, characterized in that: The third detection frame includes a fourth detection frame and a fifth detection frame, the fourth detection frame is used to mark the corresponding area of ​​the door's protective wallpaper in the target image, and the fifth detection frame is used to mark the corresponding area of ​​the window's protective wallpaper in the target image.

5. The method according to claim 4, characterized in that The determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame, and the third detection frame includes: Determine a first intersection-over-union ratio between each of the fourth detection frames and each of the first detection frames; For any of the fourth detection frames, when the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for a door; When the first intersection-and-union ratio is less than or equal to a first threshold, determining a second intersection-and-union ratio between the fourth detection frame and each of the second detection frames; When the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows; When the second intersection-over-union ratio is less than or equal to a second threshold, it is determined that the type of the corresponding protection wallpaper is other types of protection wallpaper.

6. The method according to claim 4, characterized in that The determining the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship among the first detection frame, the second detection frame and the third detection frame further includes: Determine a second intersection-over-union ratio between each of the fifth detection frames and each of the second detection frames; For any of the fifth detection frames, when the second intersection-over-union ratio is greater than a second threshold, determining that the type of the corresponding protective wallpaper is a protective wallpaper for windows; When the second intersection-and-union ratio is less than or equal to a first threshold, determining a first intersection-and-union ratio between the fifth detection frame and each of the first detection frames; When the first intersection-over-union ratio is greater than a first threshold, determining that the type of the corresponding protection wallpaper is a door protection wallpaper; When the first intersection-over-union ratio is less than or equal to a first threshold, it is determined that the type of the corresponding protection wallpaper is other types of protection wallpaper.

7. The method according to claim 4, characterized in that Before detecting and identifying the target image by using the pre-trained target detection model, the method further includes: Acquire training data and annotate the training data, wherein the annotated types include doors, windows, door protection wallpapers and window protection wallpapers, and the training data at least includes window images under a strong light environment, and the window images under a strong light environment are extracted based on a SAM image segmentation algorithm; The target detection model is trained based on the training data to obtain the pre-trained target detection model.

8. A device for identifying the type of protective wallpaper, characterized in that: include: a recognition module, configured to detect and recognize the target image through a pre-trained target detection model, and obtain a first detection frame, a second detection frame, and a third detection frame, wherein the first detection frame is used to mark the area corresponding to the door in the target image, the second detection frame is used to mark the area corresponding to the window in the target image, and the third detection frame is used to mark the area corresponding to the protective wallpaper in the target image; A determination module is used to determine the type of protective wallpaper corresponding to each third detection frame based on the relative position relationship between the first detection frame, the second detection frame and the third detection frame, and the type of protective wallpaper includes protective wallpaper for doors, standard wallpaper for windows, or other types of protective wallpaper.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images