Commodity and publicity material display style checking method, device, equipment, medium and product

Abstract

The invention discloses a commodity and propaganda material display style checking method, device and equipment, a medium and a product, and relates to the technical field of computer vision. The method comprises the following steps: firstly, carrying out feature extraction processing on a style image of a standard display style of commodities and propaganda materials to obtain propaganda material features, overall mask image features, overall segmentation image features and high-frequency commodity features of the standard display style; then propaganda material target detection processing and commodity semantic segmentation processing are carried out on the style image of the to-be-detected display style of the commodity and the propaganda material to obtain propaganda material features, overall mask graph features, overall segmentation graph features and high-frequency commodity features of the to-be-detected display style; the method comprises the following steps: firstly, comparing the multi-dimensional features of a standard display style with the multi-dimensional features of a to-be-detected display style to obtain the similarity of the two styles, and finally, determining the check result of the to-be-detected display style according to the comparison result of the similarity and a threshold value, so that the purposes of reducing the labor cost, avoiding human errors and improving the check efficiency can be achieved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and specifically relates to a method, device, equipment, medium and product for checking the display style of commodities and promotional materials. Background Art

[0002] At present, in order to highlight the promotional products during product promotion and other related activities, it is generally required to stack the promotional products in a fixed style and display them in the store with appropriate promotional materials (such as but not limited to posters, etc.). In order to unify the display styles of products and promotional materials in all chain stores and increase customer brand recognition, it is necessary to check and verify the display styles of products and promotional materials in each chain store. However, the specific method of checking and verifying the display styles of products and promotional materials is mainly to send store inspectors to the chain stores for offline inspections. This will obviously lead to problems such as high labor costs, easy human errors, and low inspection efficiency.

[0003] Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further process images so that the computer processes them into images that are more suitable for human observation or transmission to instruments for detection. Therefore, how to realize online inspection of the display styles of goods and promotional materials based on computer vision technology in order to reduce labor costs, avoid human errors and improve inspection efficiency is a topic that technicians in this field urgently need to study. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, computer equipment, computer-readable storage medium and computer program product for checking the display styles of goods and promotional materials, so as to solve the problems of high labor cost, easy human error and low checking efficiency in the existing offline checking methods of the display styles of goods and promotional materials.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for checking the display styles of commodities and promotional materials is provided, comprising:

[0007] Obtaining a style image of a standard display style of goods and promotional materials, wherein the standard display style of goods and promotional materials refers to a standard display style of goods and promotional materials;

[0008] Performing commodity and promotional material feature extraction processing on the style image of the standard display style of commodities and promotional materials, obtaining promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style of commodities and promotional materials, wherein the high-frequency commodity features include the first N commodities arranged in descending order of occurrence frequency, where N represents a positive integer greater than or equal to 2;

[0009] Obtaining a pattern image of a display pattern of commodities and promotional materials to be inspected, wherein the display pattern of commodities and promotional materials to be inspected refers to a display pattern of commodities and promotional materials to be inspected;

[0010] Performing promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected, obtaining sub-images of each promotional material in the display pattern of the goods and promotional materials to be inspected, and performing feature extraction processing on the sub-images of each promotional material to obtain the promotional material features of the display pattern of the goods and promotional materials to be inspected;

[0011] Performing commodity semantic segmentation processing on the pattern image of the commodity and promotional material display pattern to be inspected, obtaining semantic segmentation results of different commodities in the commodity and promotional material display pattern to be inspected, and counting the occurrence frequency of corresponding commodities in the commodity and promotional material display pattern to be inspected according to the semantic segmentation results, to obtain the high-frequency commodity features of the commodity and promotional material display pattern to be inspected;

[0012] Integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities to obtain an overall mask image and an overall segmentation image of the display style of the commodities and promotional materials to be inspected, perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask image features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation image features of the display style of the commodities and promotional materials to be inspected;

[0013] Compare the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, respectively, to obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected;

[0014] It is determined whether the similarity is less than a preset similarity threshold. If so, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be unqualified; otherwise, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be qualified.

[0015] Based on the above invention content, a new scheme for verifying the display styles of goods and promotional materials based on computer vision technology is provided, that is, firstly, feature extraction processing is performed on the style images of the standard display styles of goods and promotional materials to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style, and then promotional material target detection processing and commodity semantic segmentation processing are performed on the style images of the display styles of goods and promotional materials to be inspected to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the display style to be inspected, and then the multi-dimensional features of the standard display style and the display style to be inspected are compared to obtain the similarity of the two styles, and finally, according to the comparison result of the similarity and the threshold value, the verification result of the display style to be inspected is determined, so that there is no need to dispatch store inspection supervisors to the chain stores for offline inspection, thereby achieving the purpose of reducing labor costs, avoiding human errors and improving verification efficiency, and facilitating practical application and promotion.

[0016] In one possible design, a style image of a standard display style for merchandise and promotional materials is obtained, including:

[0017] Acquire a first image captured by a mobile camera when the standard display pattern of commodities and promotional materials is within a preset area of ​​interest within the field of view of the lens, and use the first image as a pattern image of the standard display pattern of commodities and promotional materials, wherein the standard display pattern of commodities and promotional materials refers to a standard display pattern of commodities and promotional materials;

[0018] Alternatively, a second image captured by a camera of the standard display style of commodities and promotional materials is first obtained, and then a labeling result of labeling an area of ​​interest of the standard display style of commodities and promotional materials in the second image is obtained in response to a human-computer interaction operation, and finally a style image of the standard display style of commodities and promotional materials is captured from the second image according to the labeling result, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials;

[0019] Alternatively, a third image of the standard display style of goods and promotional materials captured by a camera at a preset point of interest is obtained, and the third image is used as a style image of the standard display style of goods and promotional materials, wherein the point of interest refers to a camera position that can capture information of interest of the standard display style of goods and promotional materials, and the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials.

[0020] In a possible design, the pattern image of the standard display style of the commodities and promotional materials is subjected to commodity and promotional material feature extraction processing to obtain promotional material features, overall mask image features, overall segmentation image features and high-frequency commodity features of the standard display style of the commodities and promotional materials, including:

[0021] The style image of the standard display style of the goods and promotional materials is imported into the goods and promotional materials feature extraction model pre-trained based on neural network and unsupervised learning method, and the promotional material features, overall mask map features, overall segmentation map features and high-frequency goods features of the standard display style of the goods and promotional materials are output, wherein the high-frequency goods features include the first N goods arranged in descending order of occurrence frequency, and N represents a positive integer greater than or equal to 2.

[0022] In a possible design, the pattern image of the display pattern of the commodities and promotional materials to be inspected is subjected to promotional material target detection processing to obtain sub-images of each promotional material in the display pattern of the commodities and promotional materials to be inspected, including:

[0023] The pattern image of the display style of the goods and promotional materials to be inspected is imported into the promotional material recognition model pre-trained based on the YOLO target detection algorithm, and the sub-image of each promotional material in the display style of the goods and promotional materials to be inspected is output.

[0024] In a possible design, the pattern image of the display pattern of the commodities and promotional materials to be inspected is subjected to commodity semantic segmentation processing to obtain semantic segmentation results of different commodities in the display pattern of the commodities and promotional materials to be inspected, including:

[0025] The style image of the display style of the goods and promotional materials to be inspected is imported into the product semantic segmentation model obtained by pre-training based on the edge detection network and the semantic segmentation network to obtain the semantic segmentation results of different goods in the display style of the goods and promotional materials to be inspected.

[0026] In a possible design, the promotional material features, the overall mask image features, the overall segmentation image features, and the high-frequency product features of the standard display style of the goods and promotional materials are compared with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, and the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is obtained, including:

[0027] Compare the promotional material features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, and obtain the promotional material similarity S expressed by the following formula: p :

[0028]

[0029] In the formula, A p The characteristics of the promotional materials indicating the standard display style of the goods and promotional materials, B p The promotional material characteristics representing the display styles of the commodities and promotional materials to be inspected, CS() represents the cosine distance function;

[0030] By comparing the overall mask features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, the overall mask similarity S1 expressed by the following formula is obtained:

[0031] S1=CS(A1,B1)

[0032] Wherein, A1 represents the overall mask image feature of the standard display style of the goods and promotional materials, and B1 represents the overall mask image feature of the display style of the goods and promotional materials to be inspected;

[0033] By comparing the overall segmentation graph features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, the overall segmentation graph similarity S2 expressed by the following formula is obtained:

[0034] S2=CS(A2,B2)

[0035] Wherein, A2 represents the overall segmentation diagram feature of the standard display style of the commodities and promotional materials, and B1 represents the overall segmentation diagram feature of the display style of the commodities and promotional materials to be inspected;

[0036] Compare the high-frequency commodity features of the standard display style of the commodity and promotional materials with the display style of the commodity and promotional materials to be inspected to obtain the high-frequency commodity similarity S m ;

[0037] According to the similarity S of the promotional materials p , the overall mask image similarity S1, the overall segmentation image similarity S2 and the high-frequency product similarity S m , the similarity S between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is calculated according to the following formula:

[0038] S=IF(S p >S p,thres )×((η1×S1+η2×S2)+β×S m )

[0039] In the formula, S p,thresrepresents the preset promotional material similarity threshold, η1 and η2 represent weight coefficients respectively, and η1+η2=1, β represents the preset penalty coefficient, and IF() represents the indicative function.

[0040] In a second aspect, a device for checking the display style of goods and promotional materials is provided, including a style image acquisition unit, a standard feature extraction unit, a target detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit and a checking result determination unit;

[0041] The style image acquisition unit is used to acquire a style image of a standard display style of commodities and promotional materials, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials;

[0042] The standard feature extraction unit is communicatively connected to the style image acquisition unit, and is used to perform product and promotional material feature extraction processing on the style image of the standard display style of the products and promotional materials, and obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency product features of the standard display style of the products and promotional materials, wherein the high-frequency product features include the first N products arranged in descending order of occurrence frequency, where N represents a positive integer greater than or equal to 2;

[0043] The pattern image acquisition unit is further used to acquire a pattern image of a display pattern of commodities and promotional materials to be inspected, wherein the display pattern of commodities and promotional materials to be inspected refers to a display pattern of commodities / and promotional materials to be inspected;

[0044] The target detection processing unit is communicatively connected to the pattern image acquisition unit, and is used to perform promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected, obtain sub-images of each promotional material in the display pattern of the goods and promotional materials to be inspected, and perform feature extraction processing on the sub-images of each promotional material to obtain the features of the promotional materials in the display pattern of the goods and promotional materials to be inspected;

[0045] The semantic segmentation processing unit is communicatively connected to the style image acquisition unit, and is used to perform commodity semantic segmentation processing on the style image of the commodity and promotional material display style to be inspected, obtain semantic segmentation results of different commodities in the commodity and promotional material display style to be inspected, and count the occurrence frequency of the corresponding commodity in the commodity and promotional material display style to be inspected according to the semantic segmentation results, so as to obtain the high-frequency commodity features of the commodity and promotional material display style to be inspected;

[0046] The image integration processing unit is communicatively connected to the target detection processing unit and the semantic segmentation processing unit, respectively, and is used to integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities, obtain the overall mask image and the overall segmentation image of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask image features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation image features of the display style of the commodities and promotional materials to be inspected;

[0047] The style similarity determination unit is respectively communicatively connected to the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit and the image integration processing unit, and is used to respectively compare the promotional material features, the overall mask map features, the overall segmentation map features and the high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, and obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected;

[0048] The inspection result determination unit is used to determine whether the similarity is less than a preset similarity threshold. If so, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be unqualified; otherwise, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be qualified.

[0049] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for checking the display styles of goods and promotional materials as described in the first aspect or any possible design of the first aspect.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, executes the method for checking the display styles of goods and promotional materials as described in the first aspect or any possible design of the first aspect.

[0051] In a fifth aspect, the present invention provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements the method for checking the display styles of goods and promotional materials as described in the first aspect or any possible design of the first aspect.

[0052] Beneficial effects of the above scheme:

[0053] (1) The present invention provides a new solution for checking the display styles of commodities and promotional materials based on computer vision technology, that is, firstly, the style images of the standard display styles of commodities and promotional materials are subjected to feature extraction processing to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style, and then the style images of the display styles of commodities and promotional materials to be checked are subjected to promotional material target detection processing and commodity semantic segmentation processing to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the display styles to be checked, and then the multi-dimensional features of the standard display style and the display style to be checked are compared to obtain the similarity of the two styles, and finally, the verification result of the display style to be checked is determined according to the comparison result of the similarity and the threshold value, so that there is no need to dispatch store inspection supervisors to the chain stores for offline inspection, thereby achieving the purpose of reducing labor costs, avoiding human errors and improving verification efficiency, and facilitating practical application and promotion;

[0054] (2) In view of the problem that there are too many drugs and medical devices with very similar packaging in the pharmaceutical retail field, the unsupervised pre-training, semi-supervised fine-tuning and distillation created by the present invention can also improve the recognition and feature extraction of drugs, medical devices, drug-device combinations and single product posters. In addition, by combining the features and texture structures of style images, mask images and sub-images to integrate different perspective information to calculate style similarity, the fairness of the verification results is guaranteed, making it particularly suitable for the pharmaceutical retail field. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of a flow chart of a method for checking the display styles of merchandise and promotional materials provided in an embodiment of the present application.

[0057] Figure 2 This is an example diagram of a pattern image, a promotional material sub-image, an overall mask image, and an overall segmented image of the display style of the goods and promotional materials to be inspected provided in an embodiment of the present application, wherein: Figure 2 (a) in the figure shows an example of a style image of the merchandise and promotional materials to be inspected and after resizing. Figure 2 (b) in FIG. 1 shows an example of a promotional material sub-image marked with a blue mark frame in the display style image to be inspected. Figure 2(c) in the figure shows an example of the display style of the goods and promotional materials to be inspected and the integrated overall mask image. Figure 2 (d) in the figure shows an example of the overall segmented image obtained by integrating the display style of the goods and promotional materials to be inspected.

[0058] Figure 3 A schematic diagram of the structure of a device for checking the display style of goods and promotional materials provided in an embodiment of the present application.

[0059] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0061] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0062] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0063] Example

[0064] like Figure 1As shown, the commodity and promotional material display style verification method provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device with certain computing resources, such as a cloud server, a personal computer (PC, which refers to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PDA) or wearable devices. Figure 1 As shown, the method for checking the display styles of commodities and promotional materials may include, but is not limited to, the following steps S1 to S8.

[0065] S1. Obtain a style image of a standard display style of commodities and promotional materials, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials.

[0066] In step S1, the standard display style of goods and promotional materials is a standard template for checking the display style of goods and promotional materials, which can be obtained by stacking goods in a fixed style (that is, the standard display style of goods and promotional materials refers to the standard display style of goods), or by stacking goods in a fixed style and matching them with promotional materials such as posters (that is, the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials). The aforementioned promotional materials can be, but are not limited to, posters. In order to reduce background interference and improve the accuracy of subsequent verification results, it is preferred to obtain a style image of the standard display style of goods and promotional materials, including but not limited to the following methods (A) to (C).

[0067] (A) Obtaining a first image captured by a mobile camera when the standard display style of goods and promotional materials is within a preset area of ​​interest within the field of view of the lens, and using the first image as a pattern image of the standard display style of goods and promotional materials, wherein the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials. The aforementioned mobile camera may be, but is not limited to, a mobile phone camera or a tablet camera, etc., and the preset area of ​​interest may be specifically in the middle position within the field of view of the lens, similar to the central elliptical area presented on the mobile phone display screen when using a mobile phone for face recognition.

[0068] (B) First, a second image is obtained by taking a picture of the standard display style of goods and promotional materials by a camera, and then, in response to a human-computer interaction operation, a labeling result of annotating the area of ​​interest of the standard display style of goods and promotional materials in the second image is obtained, and finally, according to the labeling result, a style image of the standard display style of goods and promotional materials is obtained from the second image, wherein the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials. The aforementioned camera may be a mobile camera such as a mobile phone camera or a tablet camera, or a fixed camera such as a surveillance camera. The specific method of labeling the area of ​​interest may be, but is not limited to, a manual circle drawing method.

[0069] (C) Obtaining a third image captured by a camera at a preset point of interest of the standard display style of goods and promotional materials, and using the third image as a style image of the standard display style of goods and promotional materials, wherein the point of interest refers to a camera position that can capture interesting information of the standard display style of goods and promotional materials, and the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials. The aforementioned camera can be a mobile camera such as a mobile phone camera or a tablet camera, or a fixed camera such as a surveillance camera. The point of interest is exemplified by, but not limited to, a camera position that can capture a main view image of the standard display style of goods and promotional materials.

[0070] S2. Perform commodity and promotional material feature extraction processing on the style image of the standard display style of commodities and promotional materials to obtain promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style of commodities and promotional materials, wherein the high-frequency commodity features include the first N commodities arranged in descending order of occurrence frequency, and N represents a positive integer greater than or equal to 2.

[0071] In step S2, the promotional material features specifically include, but are not limited to, the length, width, aspect ratio, and ratio of the minimum bounding rectangle of the promotional material identified (specifically, it can be identified by a clustering algorithm) in a mask image obtained based on a pattern image of the standard display style of the commodity and promotional material (which is a binary image with the same size as the original pattern image: 1 represents the foreground pixel value, and 0 represents the background pixel value; specifically, it can be obtained by conversion using a conventional pixel threshold segmentation method). The overall mask image features specifically include, but are not limited to, the length, width, aspect ratio, and ratio of the minimum bounding rectangle of the display style identified (specifically, it can be identified by a clustering algorithm) in a mask image obtained based on a pattern image of the standard display style of the commodity and promotional material. The overall segmentation map features specifically include but are not limited to multi-dimensional feature values ​​such as the length, width, aspect ratio, and ratio of the display style area to the rectangle area of ​​the minimum circumscribed rectangle of the identified display style (which can be specifically identified by a target detection algorithm) in a segmentation image obtained based on the style image of the standard display style of the goods and promotional materials (specifically, the foreground pixel values ​​in the aforementioned mask image are replaced with the corresponding pixel values ​​in the aforementioned original style image to obtain the segmented image); generally, the overall mask map features of the standard display style of the goods and promotional materials are the same as the overall segmentation map features. The specific method for extracting and processing the features of commodities and promotional materials mentioned above can preferably be implemented by using a neural network (that is, the neural network can be regarded as a mapping relationship, an image is input, and an output feature corresponding to the input image is obtained through the mapping relationship), that is, preferably, the style image of the standard display style of the commodities and promotional materials is subjected to a commodity and promotional material feature extraction process to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style of the commodities and promotional materials, including but not limited to: importing the style image of the standard display style of the commodities and promotional materials into the commodity and promotional material feature extraction model pre-trained based on the neural network and unsupervised learning method, and outputting the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style of the commodities and promotional materials, wherein the high-frequency commodity features include the first N commodities arranged in descending order according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2. The aforementioned neural network specifically but not limited to adopting the existing Resnet50 network, and the aforementioned unsupervised learning method specifically but not limited to adopting the existing MoCo v3 training method, so as to reduce the computing power demand and training time by training the model using the self-distillation method.

[0072] S3. Obtaining a pattern image of the display pattern of the commodities and promotional materials to be inspected, wherein the display pattern of the commodities and promotional materials to be inspected refers to the display pattern of the commodities and promotional materials to be inspected.

[0073] In step S3, the display style of goods and promotional materials to be inspected is the target object for checking the display style of goods and promotional materials, which can be obtained by stacking goods in a fixed style (that is, the display style of goods and promotional materials to be inspected refers to the display style of goods to be inspected), or by stacking goods in a fixed style and matching promotional materials such as posters (that is, the display style of goods and promotional materials to be inspected refers to the display style of goods and promotional materials to be inspected), for example Figure 2 In addition, the style image of the display style of the goods and promotional materials to be inspected can be obtained by taking pictures at the chain store site using a camera such as a mobile phone camera, a tablet camera or a surveillance camera, and then uploading it to the Internet in a conventional manner.

[0074] S4. Perform promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected to obtain a sub-image of each promotional material in the display pattern of the goods and promotional materials to be inspected, and perform feature extraction processing on the sub-image of each promotional material to obtain the features of the promotional materials in the display pattern of the goods and promotional materials to be inspected.

[0075] In step S4, considering that promotional materials such as posters are relatively rare in the display style, and the style can be generated by some standard templates, and the target detection algorithm runs faster, it is preferred in this embodiment to use the target detection processing method to obtain the sub-images of each promotional material in the display style of the goods and promotional materials to be inspected. For example, Figure 2As shown in (b) in the figure. Specifically, the style image of the display style of the commodity and promotional materials to be inspected is subjected to promotional material target detection processing to obtain sub-images of each promotional material in the display style of the commodity and promotional materials to be inspected, including but not limited to: importing the style image of the display style of the commodity and promotional materials to be inspected into the promotional material recognition model pre-trained based on the YOLO target detection algorithm, and outputting the sub-images of each promotional material in the display style of the commodity and promotional materials to be inspected. The YOLO (You only look once, currently the latest version has been developed to V4, and is also widely used in the industry. Its basic principle is: first, the input image is divided into 7×7 grids, 2 borders are predicted for each grid, and then the target windows with relatively low probability are removed according to the threshold, and finally the redundant windows are removed by border merging to obtain the detection result) target detection algorithm is one of the existing target detection algorithms. The specific model structure of its V4 version consists of three parts, namely, the backbone network backbone, the neck network neck and the head network head. The backbone network Backbone can use the CSPDarknet53 (CSP stands for Cross Stage Partial) network for feature extraction. The neck network is composed of a SPP (Spatial Pyramid Pooling block) block and a PANet (Path Aggregation Network) network. The former is used to increase the receptive field and separate the most important features, and the latter is used to ensure that semantic features are received from high-level layers and fine-grained features are received from low-level layers of the horizontal backbone network at the same time. The head network head is based on anchor frame detection, and detects three different feature maps of 13×13, 26×26 and 52×52, which are used to detect targets from large to small (here, feature maps with large sizes contain richer information, so feature maps with 52×52 sizes are used to detect small targets, and vice versa). Therefore, the promotional material recognition model can be obtained by training based on a certain amount of promotional material sample images (for example, 100 poster images and 20 style transfer images corresponding to each poster image) through conventional sample training methods. In addition, the specific method of performing feature extraction processing on the sub-images of each promotional material can be obtained by referring to the conventional derivation of the aforementioned step S2, and will not be repeated here.

[0076] S5. Perform commodity semantic segmentation processing on the pattern image of the commodity and promotional materials to be inspected display pattern to obtain semantic segmentation results of different commodities in the commodity and promotional materials to be inspected display pattern, and count the occurrence frequency of corresponding commodities in the commodity and promotional materials to be inspected display pattern based on the semantic segmentation results to obtain the high-frequency commodity features of the commodity and promotional materials to be inspected display pattern.

[0077] In the step S5, semantic segmentation is an important task in computer vision, which aims to classify each pixel in the image into different semantic categories, which means marking each pixel in the image as the object or area to which it belongs, thereby achieving pixel-level classification. Since the purpose of semantic segmentation is to remove the background, reduce the interference of noisy background (i.e., distinguish the foreground from the background), and has certain fast speed and high precision characteristics, it can be applied to this embodiment to achieve the purpose of commodity semantic segmentation. In order to make full use of additional edge information so as to appropriately improve the completeness of commodity semantic segmentation, preferably, the style image of the display style of the commodity and promotional materials to be inspected is processed with commodity semantic segmentation to obtain the semantic segmentation results of different commodities in the display style of the commodity and promotional materials to be inspected, including but not limited to: importing the style image of the display style of the commodity and promotional materials to be inspected into the commodity semantic segmentation model obtained by pre-training based on the edge detection network and the semantic segmentation network, and obtaining the semantic segmentation results of different commodities in the display style of the commodity and promotional materials to be inspected. The aforementioned edge detection network and semantic segmentation network are both existing networks, so the commodity semantic segmentation model can be obtained by training based on a certain amount of commodity sample images (for example, commodity images of 9,000 individual items) through conventional sample training methods. The commodity semantic segmentation model can be trained separately for each commodity and used separately in step S5 to obtain the corresponding semantic segmentation result, or it can be trained together for all commodities and used once in step S5 to obtain the semantic segmentation results of different commodities. In addition, since the corresponding semantic segmentation results for different commodities are specifically in the form of mask images, that is, they will contain at least one mask connected area, the corresponding frequency of occurrence in the display style of the commodities and promotional materials to be inspected can be obtained by conventional statistics based on the corresponding mask connected areas (for example, equal to the number of corresponding mask connected areas or the total pixel area), and then the high-frequency commodity features of the display style of the commodities and promotional materials to be inspected can be obtained based on the commodity sorting results of the frequency of occurrence.

[0078] S6. Integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities to obtain the overall mask image and the overall segmentation image of the display style of the commodities and promotional materials to be inspected, perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask map features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation map features of the display style of the commodities and promotional materials to be inspected.

[0079] In step S6, since the corresponding semantic segmentation results for different commodities are specifically in the form of mask images, the semantic segmentation results of all commodities can be superimposed with the mask images obtained based on the sub-images of the respective promotional materials to obtain the overall mask image of the display style of the commodities and promotional materials to be inspected, for example: Figure 2 As shown in (c) in the figure; and the semantic segmentation results of all commodities, the sub-images of the respective promotional materials, and the style images of the display styles of the commodities and promotional materials to be inspected are superimposed to obtain the overall segmentation image of the display styles of the commodities and promotional materials to be inspected, for example Figure 2 In addition, the specific method of performing feature extraction processing on the overall mask image and the overall segmented image of the display style of the commodities and promotional materials to be inspected can be obtained by referring to the conventional derivation of the aforementioned step S2, which will not be repeated here.

[0080] S7. Compare the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency commodity features of the standard display style of the commodities and promotional materials with the display style of the commodities and promotional materials to be inspected one by one, respectively, to obtain the similarity between the standard display style of the commodities and promotional materials and the display style of the commodities and promotional materials to be inspected.

[0081] In the step S7, the similarity will be a pure decimal between 0 and 1 (i.e., the similarity between two images of different styles will not be 0 or 1 in general), and the larger the value, the more similar it is. The specific method of obtaining the similarity may include, but is not limited to: obtaining based on the statistical average result, or obtaining using the random forest voting mechanism in machine learning, and so on. Specifically, the promotional material features, the overall mask map features, the overall segmentation map features and the high-frequency product features of the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected are respectively compared one by one to obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected, including but not limited to the following steps S71 to S75.

[0082] S71. Compare the promotional material features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, and obtain the promotional material similarity S expressed by the following formula: p :

[0083]

[0084] In the formula, A p The characteristics of the promotional materials indicating the standard display style of the goods and promotional materials, B pThe promotional material characteristics of the commodity and promotional material to be inspected are represented by the display style, and CS() represents the cosine distance function.

[0085] In step S71, since only A p This means that there are promotional materials in the standard display style of the goods and promotional materials, but there are no promotional materials in the display style of the goods and promotional materials to be checked. Therefore, the similarity of the promotional materials S p will be zero, indicating a significant difference in the promotional material dimension. p This means that there is no promotional material in the standard display style of the goods and promotional materials, but there is promotional material in the display style of the goods and promotional materials to be inspected, so the promotional material similarity S p will also be zero to indicate a significant difference in the promotional material dimension.

[0086] S72. Compare the overall mask features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, and obtain the overall mask similarity S1 expressed by the following formula:

[0087] S1=CS(A1,B1)

[0088] In the formula, A1 represents the overall mask map feature of the standard display style of the goods and promotional materials, and B1 represents the overall mask map feature of the display style of the goods and promotional materials to be inspected.

[0089] S73. Compare the overall segmentation map features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, and obtain the overall segmentation map similarity S2 expressed by the following formula:

[0090] S2=CS(A2,B2)

[0091] In the formula, A2 represents the overall segmentation diagram characteristics of the standard display style of the goods and promotional materials, and B1 represents the overall segmentation diagram characteristics of the display style of the goods and promotional materials to be inspected.

[0092] S74. Compare the high-frequency commodity features of the standard display style of the commodity and promotional materials with the display style of the commodity and promotional materials to be inspected, and obtain the high-frequency commodity similarity S m .

[0093] In step S74, the high-frequency commodity similarity S m It can be calculated by referring to the Bleu score (which is an evaluation indicator in the field of text translation technology; in this embodiment, this indicator can be used to measure the similarity between two high-frequency product queues) or the recall rate and other calculation formulas by making regular changes.

[0094] S75. According to the similarity S of the promotional materials p , the overall mask image similarity S1, the overall segmentation image similarity S2 and the high-frequency product similarity S m , the similarity S between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is calculated according to the following formula:

[0095] S=IF(S p >S p,thres )×((η1×S1+η2×S2)+β×S m )

[0096] In the formula, S p,thres represents the preset promotional material similarity threshold, η1 and η2 represent weight coefficients respectively, and η1+η2=1, β represents the preset penalty coefficient, and IF() represents the indicative function.

[0097] S8. Determine whether the similarity is less than a preset similarity threshold. If so, determine that the inspection result of the display style of the goods and promotional materials to be inspected is unqualified; otherwise, determine that the inspection result of the display style of the goods and promotional materials to be inspected is qualified.

[0098] Therefore, based on the commodity and promotional material display style verification method described in the aforementioned steps S1 to S8, a new scheme for verifying the display style of commodities and promotional materials based on computer vision technology is provided, that is, firstly, feature extraction processing is performed on the style image of the standard display style of commodities and promotional materials to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style, and then promotional material target detection processing and commodity semantic segmentation processing are performed on the style image of the display style of commodities and promotional materials to be inspected to obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the display style to be inspected, and then the multi-dimensional features of the standard display style and the display style to be inspected are compared to obtain the similarity of the two styles, and finally, according to the comparison result of the similarity and the threshold, the verification result of the display style to be inspected is determined, so that there is no need to dispatch store inspection supervisors to the chain stores for offline inspection, thereby achieving the purpose of reducing labor costs, avoiding human errors and improving verification efficiency, and facilitating practical application and promotion. In addition, in view of the problem that there are many drugs and medical devices with very similar packaging in the pharmaceutical retail field, the unsupervised pre-training, semi-supervised fine-tuning and distillation of this embodiment can also improve the recognition and feature extraction of drugs, medical devices, drug-device combinations and single items on commodity posters. In addition, by combining the features and texture structures of style images, mask images and sub-images to integrate different perspective information to calculate style similarity, the fairness of the verification results is guaranteed, making it particularly suitable for the pharmaceutical retail field.

[0099] like Figure 3 As shown, the second aspect of this embodiment provides a virtual device for implementing the method for checking the display styles of commodities and promotional materials described in the first aspect, comprising a style image acquisition unit, a standard feature extraction unit, a target detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit and a checking result determination unit;

[0100] The style image acquisition unit is used to acquire a style image of a standard display style of commodities and promotional materials, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials;

[0101] The standard feature extraction unit is communicatively connected to the style image acquisition unit, and is used to perform product and promotional material feature extraction processing on the style image of the standard display style of the products and promotional materials, and obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency product features of the standard display style of the products and promotional materials, wherein the high-frequency product features include the first N products arranged in descending order of occurrence frequency, where N represents a positive integer greater than or equal to 2;

[0102] The pattern image acquisition unit is further used to acquire a pattern image of a display pattern of commodities and promotional materials to be inspected, wherein the display pattern of commodities and promotional materials to be inspected refers to a display pattern of commodities / and promotional materials to be inspected;

[0103] The target detection processing unit is communicatively connected to the pattern image acquisition unit, and is used to perform promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected, obtain sub-images of each promotional material in the display pattern of the goods and promotional materials to be inspected, and perform feature extraction processing on the sub-images of each promotional material to obtain the features of the promotional materials in the display pattern of the goods and promotional materials to be inspected;

[0104] The semantic segmentation processing unit is communicatively connected to the style image acquisition unit, and is used to perform commodity semantic segmentation processing on the style image of the commodity and promotional material display style to be inspected, obtain semantic segmentation results of different commodities in the commodity and promotional material display style to be inspected, and count the occurrence frequency of the corresponding commodity in the commodity and promotional material display style to be inspected according to the semantic segmentation results, so as to obtain the high-frequency commodity features of the commodity and promotional material display style to be inspected;

[0105] The image integration processing unit is communicatively connected to the target detection processing unit and the semantic segmentation processing unit, respectively, and is used to integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities, obtain the overall mask image and the overall segmentation image of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask image features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation image features of the display style of the commodities and promotional materials to be inspected;

[0106] The style similarity determination unit is respectively communicatively connected to the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit and the image integration processing unit, and is used to respectively compare the promotional material features, the overall mask map features, the overall segmentation map features and the high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, and obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected;

[0107] The inspection result determination unit is used to determine whether the similarity is less than a preset similarity threshold. If so, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be unqualified; otherwise, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be qualified.

[0108] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be found in the method for checking the display styles of goods and promotional materials described in the first aspect, and will not be described in detail here.

[0109] like Figure 4As shown, the third aspect of this embodiment provides a computer device for executing the method for checking the display style of goods and promotional materials as described in the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for checking the display style of goods and promotional materials as described in the first aspect. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0110] The working process, working details and technical effects of the aforementioned computer equipment provided in the third aspect of this embodiment can be found in the method for checking the display styles of goods and promotional materials described in the first aspect, and will not be described in detail here.

[0111] In a fourth aspect of this embodiment, there is provided a computer-readable storage medium storing instructions including the method for checking the display style of goods and promotional materials as described in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for checking the display style of goods and promotional materials as described in the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0112] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the method for checking the display styles of goods and promotional materials as described in the first aspect, and will not be described in detail here.

[0113] In a fifth aspect, the present embodiment provides a computer program product, including a computer program or an instruction, which, when executed by a computer, implements the method for checking the display styles of commodities and promotional materials as described in the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0114] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for checking the display style of commodities and promotional materials, characterized in that: include: Obtaining a style image of a standard display style of goods and promotional materials, wherein the standard display style of goods and promotional materials refers to a standard display style of goods and promotional materials; Performing commodity and promotional material feature extraction processing on the style image of the standard display style of commodities and promotional materials to obtain promotional material features, overall mask map features, overall segmentation map features and high-frequency commodity features of the standard display style of commodities and promotional materials, wherein the high-frequency commodity features include the first N commodities arranged in descending order of occurrence frequency, where N represents a positive integer greater than or equal to 2; Obtaining a pattern image of a display pattern of commodities and promotional materials to be inspected, wherein the display pattern of commodities and promotional materials to be inspected refers to a display pattern of commodities and promotional materials to be inspected; Performing promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected, obtaining sub-images of each promotional material in the display pattern of the goods and promotional materials to be inspected, and performing feature extraction processing on the sub-images of each promotional material to obtain the promotional material features of the display pattern of the goods and promotional materials to be inspected; Performing commodity semantic segmentation processing on the pattern image of the commodity and promotional material display pattern to be inspected, obtaining semantic segmentation results of different commodities in the commodity and promotional material display pattern to be inspected, and counting the occurrence frequency of corresponding commodities in the commodity and promotional material display pattern to be inspected according to the semantic segmentation results, to obtain the high-frequency commodity features of the commodity and promotional material display pattern to be inspected; Integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities to obtain an overall mask image and an overall segmentation image of the display style of the commodities and promotional materials to be inspected, perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask image features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation image features of the display style of the commodities and promotional materials to be inspected; Compare the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, respectively, to obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected; It is determined whether the similarity is less than a preset similarity threshold. If so, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be unqualified; otherwise, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be qualified.

2. The method for checking the display styles of commodities and promotional materials according to claim 1, characterized in that: Get the style images of the standard display styles of goods and promotional materials, including: Acquire a first image captured by a mobile camera when the standard display pattern of commodities and promotional materials is within a preset area of ​​interest within the field of view of the lens, and use the first image as a pattern image of the standard display pattern of commodities and promotional materials, wherein the standard display pattern of commodities and promotional materials refers to a standard display pattern of commodities and promotional materials; Alternatively, a second image captured by a camera of the standard display style of commodities and promotional materials is first obtained, and then a labeling result of labeling an area of ​​interest of the standard display style of commodities and promotional materials in the second image is obtained in response to a human-computer interaction operation, and finally a style image of the standard display style of commodities and promotional materials is captured from the second image according to the labeling result, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials; Alternatively, a third image of the standard display style of goods and promotional materials captured by a camera at a preset point of interest is obtained, and the third image is used as a style image of the standard display style of goods and promotional materials, wherein the point of interest refers to a camera position that can capture information of interest of the standard display style of goods and promotional materials, and the standard display style of goods and promotional materials refers to the standard display style of goods and promotional materials.

3. The method for checking the display styles of commodities and promotional materials according to claim 1, characterized in that: The pattern image of the standard display style of the commodities and promotional materials is subjected to commodity and promotional material feature extraction processing to obtain promotional material features, overall mask image features, overall segmentation image features and high-frequency commodity features of the standard display style of the commodities and promotional materials, including: The style image of the standard display style of the goods and promotional materials is imported into the goods and promotional materials feature extraction model pre-trained based on neural network and unsupervised learning method, and the promotional material features, overall mask map features, overall segmentation map features and high-frequency goods features of the standard display style of the goods and promotional materials are output, wherein the high-frequency goods features include the first N goods arranged in descending order of occurrence frequency, and N represents a positive integer greater than or equal to 2.

4. The method for checking the display styles of commodities and promotional materials according to claim 1, characterized in that: Performing promotional material target detection processing on the pattern image of the display pattern of the commodities and promotional materials to be inspected, and obtaining sub-images of each promotional material in the display pattern of the commodities and promotional materials to be inspected, including: The pattern image of the display style of the goods and promotional materials to be inspected is imported into the promotional material recognition model pre-trained based on the YOLO target detection algorithm, and the sub-image of each promotional material in the display style of the goods and promotional materials to be inspected is output.

5. The method for checking the display styles of commodities and promotional materials according to claim 1, characterized in that: The product semantic segmentation processing is performed on the style image of the display style of the products and promotional materials to be inspected, and the semantic segmentation results of different products in the display style of the products and promotional materials to be inspected are obtained, including: The style image of the display style of the goods and promotional materials to be inspected is imported into the product semantic segmentation model obtained by pre-training based on the edge detection network and the semantic segmentation network to obtain the semantic segmentation results of different goods in the display style of the goods and promotional materials to be inspected.

6. The method for checking the display styles of commodities and promotional materials according to claim 1, characterized in that: The promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency product features of the standard display style of the goods and promotional materials are respectively compared one by one with the display style of the goods and promotional materials to be inspected, and the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is obtained, including: Compare the promotional material features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, and obtain the promotional material similarity S expressed by the following formula: p : In the formula, A p The characteristics of the promotional materials indicating the standard display style of the goods and promotional materials, B p The promotional material characteristics representing the display styles of the commodities and promotional materials to be inspected, CS() represents the cosine distance function; By comparing the overall mask features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, the overall mask similarity S1 expressed by the following formula is obtained: S1=CS(A1,B1) Wherein, A1 represents the overall mask image feature of the standard display style of the goods and promotional materials, and B1 represents the overall mask image feature of the display style of the goods and promotional materials to be inspected; By comparing the overall segmentation graph features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, the overall segmentation graph similarity S2 expressed by the following formula is obtained: S2=CS(A2,B2) Wherein, A2 represents the overall segmentation diagram feature of the standard display style of the commodities and promotional materials, and B1 represents the overall segmentation diagram feature of the display style of the commodities and promotional materials to be inspected; Compare the high-frequency commodity features of the standard display style of the commodity and promotional materials with the display style of the commodity and promotional materials to be inspected to obtain the high-frequency commodity similarity S m ; According to the similarity S of the promotional materials p , the overall mask image similarity S1, the overall segmentation image similarity S2 and the high-frequency product similarity S m , the similarity S between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is calculated according to the following formula: S=IF(S p >S p,thres )×((η1×S1+η2×S2)+β×S m ) In the formula, S p,thres represents the preset promotional material similarity threshold, η1 and η2 represent weight coefficients respectively, and η1+η2=1, β represents the preset penalty coefficient, and IF() represents the indicative function.

7. A device for checking the display style of commodities and promotional materials, characterized in that: It includes a style image acquisition unit, a standard feature extraction unit, a target detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit and a verification result determination unit; The style image acquisition unit is used to acquire a style image of a standard display style of commodities and promotional materials, wherein the standard display style of commodities and promotional materials refers to a standard display style of commodities and promotional materials; The standard feature extraction unit is communicatively connected to the style image acquisition unit, and is used to perform product and promotional material feature extraction processing on the style image of the standard display style of the products and promotional materials, and obtain the promotional material features, overall mask map features, overall segmentation map features and high-frequency product features of the standard display style of the products and promotional materials, wherein the high-frequency product features include the first N products arranged in descending order of occurrence frequency, where N represents a positive integer greater than or equal to 2; The pattern image acquisition unit is further used to acquire a pattern image of a display pattern of commodities and promotional materials to be inspected, wherein the display pattern of commodities and promotional materials to be inspected refers to a display pattern of commodities / and promotional materials to be inspected; The target detection processing unit is communicatively connected to the pattern image acquisition unit, and is used to perform promotional material target detection processing on the pattern image of the display pattern of the goods and promotional materials to be inspected, obtain sub-images of each promotional material in the display pattern of the goods and promotional materials to be inspected, and perform feature extraction processing on the sub-images of each promotional material to obtain the features of the promotional materials in the display pattern of the goods and promotional materials to be inspected; The semantic segmentation processing unit is communicatively connected to the style image acquisition unit, and is used to perform commodity semantic segmentation processing on the style image of the commodity and promotional material display style to be inspected, obtain semantic segmentation results of different commodities in the commodity and promotional material display style to be inspected, and count the occurrence frequency of the corresponding commodity in the commodity and promotional material display style to be inspected according to the semantic segmentation results, so as to obtain the high-frequency commodity features of the commodity and promotional material display style to be inspected; The image integration processing unit is communicatively connected to the target detection processing unit and the semantic segmentation processing unit, respectively, and is used to integrate the sub-images of the various promotional materials and the semantic segmentation results of the different commodities, obtain the overall mask image and the overall segmentation image of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall mask image of the display style of the commodities and promotional materials to be inspected to obtain the overall mask image features of the display style of the commodities and promotional materials to be inspected, and perform feature extraction processing on the overall segmentation image of the display style of the commodities and promotional materials to be inspected to obtain the overall segmentation image features of the display style of the commodities and promotional materials to be inspected; The style similarity determination unit is respectively communicatively connected to the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit and the image integration processing unit, and is used to respectively compare the promotional material features, the overall mask map features, the overall segmentation map features and the high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected in a one-to-one correspondence, and obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected; The inspection result determination unit is used to determine whether the similarity is less than a preset similarity threshold. If so, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be unqualified; otherwise, the inspection result of the display style of the goods and promotional materials to be inspected is determined to be qualified.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for checking the display style of goods and promotional materials as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the method for checking the display style of goods and promotional materials as described in any one of claims 1 to 6 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the method for checking the display styles of commodities and promotional materials as claimed in any one of claims 1 to 6 is implemented.

Images