Commodity and promotional material display style checking method, device, equipment, medium and product

Abstract

The application discloses a kind of commodity and publicity material display style checking method, device, equipment, medium and product, it is related to computer vision technical field.The method is first to the style image of commodity and publicity material standard display style Feature extraction processing, obtain the publicity material feature of standard display style, overall mask chart feature, overall segmentation chart feature and high-frequency commodity feature, then to the style image of commodity and publicity material to be checked display style Publicity material target detection processing and commodity semantic segmentation processing, obtain the publicity material feature of to be checked display style, overall mask chart feature, overall segmentation chart feature and high-frequency commodity feature, then compare the multidimensional feature of standard display style and to be checked display style, obtain the similarity of two styles, finally according to the comparison result of similarity and threshold value, determine the checking result of to be checked display style, so it can realize the purpose of reducing labor cost, avoiding human error and improving checking efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a commodity and promotional material display style checking method, device, equipment, medium and product. BACKGROUND

[0002] At present, in order to highlight the activity commodities, it is generally required to stack and place the activity commodities into a fixed style and possibly match appropriate promotional materials (such as but not limited to posters, etc.) for display in the store during the commodity promotion and other related activities. In order to unify the commodity and promotional material display styles of all chain stores and increase the brand recognition of customers, it is necessary to check and compare the commodity and promotional material display styles of each chain store. However, the existing specific way of checking and comparing the commodity and promotional material display styles mainly sends on-site offline inspection of the chain stores by the store inspection supervisors, which obviously has the problems of high labor cost, easy human error and low checking efficiency.

[0003] Computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets and further perform image processing, so that the computer processing becomes images more suitable for human observation or transmission to instruments for detection. Therefore, how to realize online inspection of the commodity and promotional material display styles based on computer vision technology in order to reduce labor cost, avoid human error and improve checking efficiency is a subject that the technical personnel in the field urgently need to study. SUMMARY

[0004] The purpose of the present application is to provide a commodity and promotional material display style checking method, device, computer equipment, computer readable storage medium and computer program product, which can solve the problems of high labor cost, easy human error and low checking efficiency of the existing offline checking method of the commodity and promotional material display styles.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, a commodity and promotional material display style checking method is provided, comprising:

[0007] Obtaining a style image of a commodity and promotional material standard display style, wherein the commodity and promotional material standard display style refers to a standard display style of commodities and / or promotional materials;

[0008] extracting features of the product and promotional material from the style image of the standard product and promotional material display style to obtain promotional material features, overall mask image features, overall segmentation image features and high-frequency product features of the standard product and promotional material display style, wherein the high-frequency product features include the top N products arranged in order from high to low according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2;

[0009] obtaining a style image of a product and promotional material to be inspected display style, wherein the product and promotional material to be inspected display style refers to a to-be-inspected display style of products and / or promotional materials;

[0010] detecting promotional material targets from the style image of the product and promotional material to be inspected display style to obtain sub-images of each promotional material in the product and promotional material to be inspected display style, and extracting features from the sub-images of each promotional material to obtain the promotional material features of the product and promotional material to be inspected display style;

[0011] performing semantic segmentation of products on the style image of the product and promotional material to be inspected display style to obtain semantic segmentation results of different products in the product and promotional material to be inspected display style, and counting the frequency of occurrence of corresponding products in the product and promotional material to be inspected display style according to the semantic segmentation results to obtain the high-frequency product features of the product and promotional material to be inspected display style;

[0012] integrating the sub-images of each promotional material and the semantic segmentation results of different products to obtain an overall mask image and an overall segmentation image of the product and promotional material to be inspected display style, extracting features from the overall mask image of the product and promotional material to be inspected display style to obtain the overall mask image features of the product and promotional material to be inspected display style, and extracting features from the overall segmentation image of the product and promotional material to be inspected display style to obtain the overall segmentation image features of the product and promotional material to be inspected display style;

[0013] respectively and one by one, comparing the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency product features of the product and promotional material standard display style and the product and promotional material to be inspected display style to obtain the similarity between the product and promotional material standard display style and the product and promotional material to be inspected display style;

[0014] determining whether the similarity is less than a preset similarity threshold, if yes, determining that the inspection result of the product and promotional material to be inspected display style is unqualified, otherwise determining that the inspection result of the product and promotional material to be inspected display style is qualified.

[0015] Based on the above invention content, a new scheme for checking the display style of goods and promotional materials based on computer vision technology is provided, that is, first, the feature extraction processing is performed on the style image of the standard display style of goods and promotional materials to obtain the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency goods features of the standard display style, then the promotional material target detection processing and the goods semantic segmentation processing are performed on the style image of the to-be-checked display style of goods and promotional materials to obtain the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency goods features of the to-be-checked display style, then the multi-dimensional features of the standard display style and the to-be-checked display style are compared to obtain the similarity of the two styles, and finally, according to the comparison result of the similarity and the threshold value, the checking result of the to-be-checked display style is determined, so that the on-site offline inspection of the chain store by the store inspection supervisor can be avoided, the purpose of reducing labor cost, avoiding human error and improving checking efficiency is achieved, and the actual application and popularization are facilitated.

[0016] In one possible design, the style image of the standard display style of goods and promotional materials is obtained, including:

[0017] The first image obtained by the mobile camera when the standard display style of goods and promotional materials is in the preset region of interest in the lens field of view is obtained, and the first image is taken as the style image of the standard display style of goods and promotional materials, wherein the standard display style of goods and promotional materials refers to the standard display style of goods and / or promotional materials.

[0018] Alternatively, the second image obtained by the camera when the standard display style of goods and promotional materials is photographed is obtained, and then the annotation result of the region of interest in the second image for the standard display style of goods and promotional materials is obtained in response to the human-computer interaction operation, and finally the style image of the standard display style of goods and promotional materials is obtained from the second image according to the annotation result, wherein the standard display style of goods and promotional materials refers to the standard display style of goods and / or promotional materials.

[0019] Alternatively, the third image obtained by the camera when the standard display style of goods and promotional materials is photographed at the preset point of interest is obtained, and the third image is taken as the style image of the standard display style of goods and promotional materials, wherein the point of interest refers to the camera position that can obtain the interested information of the standard display style of goods and promotional materials, and the standard display style of goods and promotional materials refers to the standard display style of goods and / or promotional materials.

[0020] In a possible design, the style image of the standard display style of the commodity and promotional material is subjected to commodity and promotional material feature extraction processing to obtain promotional material features, an overall mask map feature, an overall segmentation map feature, and high-frequency commodity features of the standard display style of the commodity and promotional material, including:

[0021] The style image of the standard display style of the commodity and promotional material is input into a commodity and promotional material feature extraction model pre-trained based on a neural network and an unsupervised learning manner, and promotional material features, an overall mask map feature, an overall segmentation map feature, and high-frequency commodity features of the standard display style of the commodity and promotional material are output, where the high-frequency commodity features include the first N commodities arranged in order from high to low according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2.

[0022] In a possible design, the style image of the to-be-inspected display style of the commodity and promotional material is subjected to promotional material target detection processing to obtain sub-images of each promotional material in the to-be-inspected display style of the commodity and promotional material, including:

[0023] The style image of the to-be-inspected display style of the commodity and promotional material is input into a promotional material recognition model pre-trained based on a YOLO target detection algorithm, and sub-images of each promotional material in the to-be-inspected display style of the commodity and promotional material are output.

[0024] In a possible design, the style image of the to-be-inspected display style of the commodity and promotional material is subjected to commodity semantic segmentation processing to obtain semantic segmentation results of different commodities in the to-be-inspected display style of the commodity and promotional material, including:

[0025] The style image of the to-be-inspected display style of the commodity and promotional material is input into a commodity semantic segmentation model pre-trained based on an edge detection network and a semantic segmentation network, and semantic segmentation results of different commodities in the to-be-inspected display style of the commodity and promotional material are obtained.

[0026] In a possible design, the promotional material features, the overall mask map features, the overall segmentation map features, and the high-frequency commodity features of the standard display style of the commodity and promotional material and the to-be-inspected display style of the commodity and promotional material are respectively and one-to-one compared to obtain the similarity between the standard display style of the commodity and promotional material and the to-be-inspected display style of the commodity and promotional material, including:

[0027] The promotional material features of the standard display style of the commodity and promotional material and the to-be-inspected display style of the commodity and promotional material are compared to obtain a promotional material similarity S expressed by the following formula: p

[0028] ​

[0029] In the formula, A p represents the promotional material features of the standard display style of the commodity and promotional material, B p represents the promotional material features of the display style to be inspected of the commodity and promotional material, CS() represents a cosine distance function;

[0030] By comparing the overall mask image features of the standard display style of the commodity and promotional material and the display style to be inspected of the commodity and promotional material, an overall mask image similarity S1 is obtained, which is expressed by the following formula:

[0031] S1 = CS(A1, B1)

[0032] In the formula, A1 represents the overall mask image features of the standard display style of the commodity and promotional material, and B1 represents the overall mask image features of the display style to be inspected of the commodity and promotional material;

[0033] By comparing the overall segmentation image features of the standard display style of the commodity and promotional material and the display style to be inspected of the commodity and promotional material, an overall segmentation image similarity S2 is obtained, which is expressed by the following formula:

[0034] S2 = CS(A2, B2)

[0035] In the formula, A2 represents the overall segmentation image features of the standard display style of the commodity and promotional material, and B1 represents the overall segmentation image features of the display style to be inspected of the commodity and promotional material;

[0036] By comparing the high-frequency commodity features of the standard display style of the commodity and promotional material and the display style to be inspected of the commodity and promotional material, a high-frequency commodity similarity S m is obtained.

[0037] According to the promotional material similarity S p , the overall mask image similarity S1, the overall segmentation image similarity S2 and the high-frequency commodity similarity S m , a similarity S between the standard display style of the commodity and promotional material and the display style to be inspected of the commodity and promotional material is calculated according to the following formula:

[0038] S = IF(S p > S p,thres ) × ((η1 × S1 + η2 × S2) + β × S m )

[0039] In the formula, S p,thresη1 and η2 represent weight coefficients respectively and have η1 + η2 = 1, β represents a preset penalty coefficient, and IF() represents an indicative function.

[0040] In a second aspect, a commodity and promotional material display style checking device is provided, comprising a style image acquisition unit, a standard feature extraction unit, a target detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit, and a checking result determination unit.

[0041] The style image acquisition unit is configured to acquire a style image of a commodity and promotional material standard display style, wherein the commodity and promotional material standard display style refers to a standard display style of commodities and promotional materials.

[0042] The standard feature extraction unit is in communication connection with the style image acquisition unit and is configured to perform commodity and promotional material feature extraction processing on the style image of the commodity and promotional material standard display style to obtain promotional material features, an overall mask feature, an overall segmentation feature, and high-frequency commodity features of the commodity and promotional material standard display style, wherein the high-frequency commodity features contain the first N commodities arranged in order from high to low according to the appearance frequency, and N represents a positive integer greater than or equal to 2.

[0043] The style image acquisition unit is further configured to acquire a style image of a commodity and promotional material to-be-checked display style, wherein the commodity and promotional material to-be-checked display style refers to a to-be-checked display style of commodities and promotional materials.

[0044] The target detection processing unit is in communication connection with the style image acquisition unit and is configured to perform promotional material target detection processing on the style image of the commodity and promotional material to-be-checked display style to obtain sub-images of each promotional material in the commodity and promotional material to-be-checked display style, and perform feature extraction processing on the sub-images of each promotional material to obtain the promotional material features of the commodity and promotional material to-be-checked display style.

[0045] The semantic segmentation processing unit is in communication connection with the style image acquisition unit and is configured to perform commodity semantic segmentation processing on the style image of the commodity and promotional material to-be-checked display style to obtain semantic segmentation results of different commodities in the commodity and promotional material to-be-checked display style, and count the appearance frequency of a corresponding commodity in the commodity and promotional material to-be-checked display style according to the semantic segmentation results to obtain the high-frequency commodity features of the commodity and promotional material to-be-checked display style.

[0046] The image integration processing unit is respectively communicatively connected with the target detection processing unit and the semantic segmentation processing unit, and is configured to integrate the sub-image of each promotional material and the semantic segmentation result of different commodities, to obtain an overall mask image and an overall segmentation image of the to-be-inspected display style of the commodities and the promotional materials, to perform feature extraction processing on the overall mask image of the to-be-inspected display style of the commodities and the promotional materials, to obtain an overall mask image feature of the to-be-inspected display style of the commodities and the promotional materials, and to perform feature extraction processing on the overall segmentation image of the to-be-inspected display style of the commodities and the promotional materials, to obtain an overall segmentation image feature of the to-be-inspected display style of the commodities and the promotional materials.

[0047] The style similarity determination unit is respectively communicatively connected with the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit and the image integration processing unit, and is configured to one-to-one correspondingly compare the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency commodity features of the standard display style of the commodities and the promotional materials and the to-be-inspected display style of the commodities and the promotional materials, to obtain a similarity between the standard display style of the commodities and the promotional materials and the to-be-inspected display style of the commodities and the promotional materials.

[0048] The inspection result determination unit is configured to determine whether the similarity is less than a preset similarity threshold value, and if yes, to determine that an inspection result of the to-be-inspected display style of the commodities and the promotional materials is unqualified, and if not, to determine that the inspection result of the to-be-inspected display style of the commodities and the promotional materials is qualified.

[0049] In a third aspect, the present application provides a computer device, comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to transceive messages, and the processor is configured to read the computer program and execute the display style inspection method of the commodities and the promotional materials according to any possible design of the first aspect.

[0050] In a fourth aspect, the present application provides a computer readable storage medium, wherein instructions are stored on the computer readable storage medium, and when the instructions are executed on a computer, the display style inspection method of the commodities and the promotional materials according to any possible design of the first aspect is executed.

[0051] In a fifth aspect, the present application provides a computer program product, comprising a computer program or instructions, and when the computer program or the instructions are executed on a computer, the display style inspection method of the commodities and the promotional materials according to any possible design of the first aspect is implemented.

[0052] The above-mentioned scheme has the following beneficial effects:

[0053] (1) The application provides a new scheme for checking the display style of goods and promotional materials based on computer vision technology, that is, first, the feature extraction processing is performed on the style image of the standard display style of goods and promotional materials to obtain the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency goods features of the standard display style, then the promotional material target detection processing and the goods semantic segmentation processing are performed on the style image of the to-be-checked display style of goods and promotional materials to obtain the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency goods features of the to-be-checked display style, then the multi-dimensional features of the standard display style and the to-be-checked display style are compared to obtain the similarity of the two styles, and finally, the checking result of the to-be-checked display style is determined according to the comparison result of the similarity and the threshold value, so that the on-site offline inspection of the chain stores by the store inspection supervisors can be avoided, the labor cost is reduced, the human error is avoided and the checking efficiency is improved, and the application and popularization are facilitated.

[0054] (2) In view of the problem that there are many drugs and medical devices in the medical retail field and the packaging is very similar, the unsupervised pre-training, semi-supervised fine-tuning and distillation of the application can also improve the recognition and feature extraction effect of drugs, medical devices, drug and medical device combinations and commodity poster single products, and the fairness of the checking result is ensured by combining the features / texture structures of the style image, the mask image and the sub-image to integrate different visual angle information to calculate the style similarity, and the application is particularly suitable for the medical retail field. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0056] Figure 1 The flowchart of the display style checking method of goods and promotional materials provided by the embodiments of the application.

[0057] Figure 2 The example diagrams of the style image, the promotional material sub-image, the overall mask image and the overall segmentation image of the to-be-checked display style of goods and promotional materials provided by the embodiments of the application are shown in the following table. Figure 2 (a) in the table shows an example diagram of the style image of the to-be-checked display style of goods and promotional materials after size adjustment, Figure 2 (b) in the table shows an example diagram of the promotional material sub-image marked with a blue marked box in the to-be-checked display style image, Figure 2(c) in FIG. 1 shows an example diagram of the merchandise and promotional material to be inspected display style and the integrated overall mask image, Figure 2 (d) in FIG. 1 shows an example diagram of the merchandise and promotional material to be inspected display style and the integrated overall segmentation image.

[0058] Figure 3 A structural schematic diagram of a merchandise and promotional material display style inspection device provided by an embodiment of the present application.

[0059] Figure 4 A structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the present application will be briefly introduced below with reference to the drawings and the description of the embodiments or the prior art. Obviously, the following description of the drawings is only some embodiments of the present application, and for those skilled in the art, other embodiments can be obtained without creative labor on the basis of these embodiments. It should be noted that the description of these embodiments is used to help understand the present application, but does not constitute a limitation on the present application.

[0061] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object, without departing from the scope of the example embodiments of the present application.

[0062] It should be understood that for the term "and / or" that may appear in the present text, it is only a description of the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can mean that A exists alone, B exists alone, or A and B exist together, etc. For example, A, B and / or C, which means that any one of A, B and C or any combination thereof exists; for the term " / and" that may appear in the present text, it is another description of the relationship of another associated object, which means that there can be two kinds of relationships, for example, A / and B, which means that A exists alone or A and B exist together, etc. In addition, for the character " / " that may appear in the present text, it generally means that the associated objects before and after are an "or" relationship.

[0063] EMBODIMENT

[0064] As Figure 1As shown, the method for checking the display style of the commodity and promotional materials provided by the first aspect of the present embodiment can be, but is not limited to, executed by a computer device with certain computing resources, such as a cloud server, a personal computer (PC, which refers to a multi-purpose computer suitable for personal use in size, price and performance; desktop computers, notebook computers to small notebook computers and tablet computers, and ultrabooks, etc.), a smart phone, a personal digital assistant (PDA), or a wearable device, etc. electronic equipment. For example, Figure 1 As shown, the method for checking the display style of the commodity and promotional materials can include, but is not limited to, the following steps S1-S8.

[0065] S1. Obtain the style image of the standard display style of the commodity and promotional materials, wherein the standard display style of the commodity and promotional materials refers to the standard display style of the commodity and promotional materials.

[0066] In the step S1, the standard display style of the commodity and promotional materials is a standard template for checking the display style of the commodity and promotional materials, which can be obtained by stacking and placing the commodity in a fixed style (i.e., the standard display style of the commodity and promotional materials refers to the standard display style of the commodity), or by stacking and placing the commodity in a fixed style and matching with promotional materials such as posters (i.e., the standard display style of the commodity and promotional materials refers to the standard display style of the commodity and promotional materials). The aforementioned promotional materials can be, but are not limited to, posters. In order to reduce background interference and improve the accuracy of subsequent checking results, preferably, the style image of the standard display style of the commodity and promotional materials is obtained, including but not limited to the following ways (A)-(C).

[0067] (A) Obtain a first image obtained by a mobile camera when the standard display style of the commodity and promotional materials is in a preset region of interest within the lens field of view, and take the first image as the style image of the standard display style of the commodity and promotional materials, wherein the standard display style of the commodity and promotional materials refers to the standard display style of the commodity and promotional materials. The aforementioned mobile camera can be, but is not limited to, a mobile phone camera or a tablet computer camera, and the preset region of interest can be specifically in the middle position of the lens field of view, similar to the central oval region presented on the display screen of the mobile phone when using the mobile phone for face recognition.

[0068] (B) first acquiring a second image obtained by a camera shooting a standard display style of a commodity and a promotional material, then responding to a human-computer interaction operation to obtain a labeling result of labeling a region of interest of the standard display style of the commodity and the promotional material in the second image, and finally obtaining a style image of the standard display style of the commodity and the promotional material from the second image according to the labeling result. The standard display style of the commodity and the promotional material refers to a standard display style of a commodity and / or a promotional material. The camera can be a mobile camera such as a mobile phone camera or a tablet computer camera, or a fixed camera such as a surveillance camera. The specific way of labeling the region of interest can be, but is not limited to, manually drawing a circle.

[0069] (C) acquiring a third image obtained by a camera shooting a standard display style of a commodity and a promotional material at a preset point of interest, and taking the third image as a style image of the standard display style of the commodity and the promotional material. The point of interest refers to a camera position that can shoot the interested information of the standard display style of the commodity and the promotional material. The standard display style of the commodity and the promotional material refers to a standard display style of a commodity and / or a promotional material. The camera can be a mobile camera such as a mobile phone camera or a tablet computer camera, or a fixed camera such as a surveillance camera. The point of interest is exemplified but not limited to a camera position that can shoot a front view image of the standard display style of the commodity and the promotional material.

[0070] S2. performing a commodity and promotional material feature extraction process on the style image of the standard display style of the commodity and the promotional material to obtain a promotional material feature, an overall mask image feature, an overall segmentation image feature and a high-frequency commodity feature of the standard display style of the commodity and the promotional material. The high-frequency commodity feature contains the first N commodities arranged in order from high to low according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2.

[0071] In the step S2, the promotional material features specifically but not limited to include the length, width, aspect ratio of the minimum bounding rectangle of the identified promotional material (specifically identified by clustering algorithm) and the ratio of the promotional material area to the rectangle area, etc. multi-dimensional feature values in the mask image obtained based on the style image of the standard display style of the goods and promotional materials. The overall mask image features specifically but not limited to include the length, width, aspect ratio of the minimum bounding rectangle of the identified display style (specifically identified by clustering algorithm) and the ratio of the display style area to the rectangle area, etc. multi-dimensional feature values in the mask image obtained based on the style image of the standard display style of the goods and promotional materials. The overall segmentation image features specifically but not limited to include the length, width, aspect ratio of the minimum bounding rectangle of the identified display style (specifically identified by target detection algorithm) and the ratio of the display style area to the rectangle area, etc. multi-dimensional feature values in the segmentation image (specifically replace the foreground pixel value in the aforementioned mask image with the corresponding pixel value in the aforementioned original style image, and the segmentation image can be obtained) obtained based on the style image of the standard display style of the goods and promotional materials. Generally, the overall mask image features and the overall segmentation image features of the standard display style of the goods and promotional materials are the same. The specific way of the aforementioned feature extraction processing of the goods and promotional materials can preferably be realized by using a neural network (i.e. the neural network can be regarded as a mapping relationship, the input image is obtained through the mapping relationship, and the output features corresponding to the input image are obtained), that is, preferably, the style image of the standard display style of the goods and promotional materials is subjected to goods and promotional material feature extraction processing to obtain the promotional material features, overall mask image features, overall segmentation image features and high-frequency goods features of the standard display style of the goods and promotional materials, including but not limited to: importing the style image of the standard display style of the goods and promotional materials into the goods and promotional material feature extraction model pre-trained based on the neural network and unsupervised learning method, and outputting the promotional material features, overall mask image features, overall segmentation image features and high-frequency goods features of the standard display style of the goods and promotional materials, wherein the high-frequency goods features include the top N goods arranged in order from high to low according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2. The aforementioned neural network specifically but not limited to uses the existing Resnet50 network, and the aforementioned unsupervised learning method specifically but not limited to uses the existing MoCo v3 training method, so as to realize the purpose of reducing the demand for computing power and training time by using the self-distillation method to train the model.

[0072] S3. Obtain a style image of the commodity and promotional material to be inspected and displayed, wherein the commodity and promotional material to be inspected and displayed refers to the commodity and promotional material to be inspected and displayed.

[0073] In the step S3, the commodity and promotional material to be inspected and displayed is the target object for the commodity and promotional material display style inspection, which can be obtained by stacking and placing the commodity in a fixed style (i.e., the commodity and promotional material to be inspected and displayed refers to the commodity to be inspected and displayed), or by stacking and placing the commodity in a fixed style and combining with promotional materials such as posters (i.e., the commodity and promotional material to be inspected and displayed refers to the commodity and promotional material to be inspected and displayed), for example, as shown in (a) of FIG. 1. In addition, the style image of the commodity and promotional material to be inspected and displayed can be obtained by using a camera such as a mobile phone camera, a tablet computer camera or a monitoring camera to take pictures on the spot of a chain store, and then uploaded based on the Internet to obtain. Figure 2

[0074] S4. Perform promotional material target detection processing on the style image of the commodity and promotional material to be inspected and displayed, obtain a sub-image of each promotional material in the commodity and promotional material to be inspected and displayed, and perform feature extraction processing on the sub-image of each promotional material to obtain the promotional material features of the commodity and promotional material to be inspected and displayed.

[0075] In the step S4, considering that promotional materials such as posters are relatively less in the display style, and the style can be generated by some standard templates, and the running speed of the target detection algorithm is fast, in this embodiment, the target detection processing mode is preferably used to obtain the sub-image of each promotional material in the commodity and promotional material to be inspected and displayed, for example, as shown in (b) of FIG. 1. Figure 2 ​(b) shown in the above step S4. Specifically, the style image of the to-be-inspected display style of the commodity and promotional material is subjected to promotional material target detection processing to obtain sub-images of each promotional material in the to-be-inspected display style of the commodity and promotional material, including but not limited to: importing the style image of the to-be-inspected display style of the commodity and promotional material into a promotional material recognition model pre-trained based on a YOLO target detection algorithm to output sub-images of each promotional material in the to-be-inspected display style of the commodity and promotional material. The YOLO (You only look once, the latest version has developed to V4, which is widely used in the industry, and its basic principle is: first, divide the input image into a 7x7 grid, predict 2 bounding boxes for each grid, then remove the target window with low possibility according to the threshold, and finally remove the redundant window using the bounding box merging method to obtain the detection result) target detection algorithm is one of the existing target detection algorithms, and the specific model structure of the V4 version is composed of three parts, namely the backbone network, the neck network and the head network. The backbone network Backbone can adopt a CSPDarknet53 (CSP represents Cross Stage Partial) network for feature extraction. The neck network neck is composed of an SPP (Spatial Pyramid Pooling block) block and a PANet (Path Aggregation Network) network, the former is used to increase the receptive field and separate the most important features, and the latter is used to ensure that semantic features are received from high-level layers and fine-grained features are received from low-level layers of the horizontal backbone network. The head network head is based on anchor box detection and detects three different size feature maps of 13x13, 26x26 and 52x52, which are used to detect large to small targets (here, the larger the feature map size, the more information it contains, therefore, the 52x52 size feature map is used to detect small targets, and vice versa). Therefore, based on a certain amount of promotional material sample images (for example, 100 poster images and 20 style transfer images corresponding to each poster image), the promotional material recognition model can be trained by conventional sample training method. In addition, the specific way of feature extraction processing of the sub-images of each promotional material can be derived conventionally as described in the foregoing step S2, and will not be described here.

[0076] S5. The style image of the to-be-inspected display style of the commodity and promotional material is subjected to commodity semantic segmentation processing to obtain semantic segmentation results of different commodities in the to-be-inspected display style of the commodity and promotional material, and the occurrence frequency of the corresponding commodity in the to-be-inspected display style of the commodity and promotional material is counted based on the semantic segmentation results to obtain the high-frequency commodity features of the to-be-inspected display style of the commodity and promotional material.

[0077] In the step S5, semantic segmentation is an important task in computer vision, which aims to classify each pixel in an image into different semantic categories, which means labeling each pixel in the image as the object or region it belongs to, thereby achieving pixel-level classification. Since the purpose of semantic segmentation is to remove the background, reduce the interference of noisy background (i.e. distinguish foreground from background), and have certain speed and high precision characteristics, it can be applied to the present embodiment to achieve the purpose of commodity semantic segmentation. In order to fully use the additional edge information in order to properly improve the completeness of commodity semantic segmentation, preferably, the style image of the commodity and promotional material to be inspected display style is subjected to commodity semantic segmentation processing to obtain the semantic segmentation result of different commodities in the commodity and promotional material to be inspected display style, including but not limited to: importing the style image of the commodity and promotional material to be inspected display style into the commodity semantic segmentation model obtained by pre-training based on the edge detection network and the semantic segmentation network to obtain the semantic segmentation result of different commodities in the commodity and promotional material to be inspected display style. The aforementioned edge detection network and semantic segmentation network are both existing networks, so the commodity semantic segmentation model can be trained based on a certain amount of commodity sample images (such as 9000 single product commodity images) by conventional sample training method. The commodity semantic segmentation model can be trained separately for each commodity and used separately in step S5 to obtain the corresponding semantic segmentation result, or it can be trained together for all commodities and used once in step S5 to obtain the semantic segmentation result of different commodities. In addition, since the corresponding semantic segmentation result for different commodities is in the form of a mask image, which will contain at least one mask connected region, the corresponding appearance frequency (such as the number of corresponding mask connected regions or total pixel area) in the commodity and promotional material to be inspected display style can be obtained by conventional statistics based on the corresponding mask connected region, and the high-frequency commodity feature of the commodity and promotional material to be inspected display style can be obtained based on the commodity ranking result of the appearance frequency.

[0078] S6. Integrating the sub-images of the various promotional materials and the semantic segmentation results of the different commodities, obtaining the overall mask image and the overall segmentation image of the commodity and promotional material to be inspected display style, and performing feature extraction processing on the overall mask image of the commodity and promotional material to be inspected display style to obtain the overall mask image feature of the commodity and promotional material to be inspected display style, and performing feature extraction processing on the overall segmentation image of the commodity and promotional material to be inspected display style to obtain the overall segmentation image feature of the commodity and promotional material to be inspected display style.

[0079] In step S6, since the semantic segmentation results for different products are specifically in the form of mask images, the semantic segmentation results of all products can be superimposed with the mask images obtained based on the sub-images of each promotional material to obtain the overall mask image of the product and promotional material display style to be inspected. For example... Figure 2 As shown in (c); and by overlaying the semantic segmentation results of all products, the sub-images of each promotional material, and the style image of the product and promotional material display style to be inspected, the overall segmentation image of the product and promotional material display style to be inspected is obtained, for example... Figure 2 As shown in (d) above. Furthermore, the specific method for feature extraction processing of the overall mask image and overall segmented image of the product and promotional materials to be inspected can be derived conventionally with reference to the aforementioned step S2, and will not be repeated here.

[0080] S7. Compare the promotional material features, overall mask image features, overall segmentation image features, and high-frequency product features of the standard display style of the goods and promotional materials with the display style of the goods and promotional materials to be inspected, one by one, to obtain the similarity between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected.

[0081] In step S7, the similarity score is a decimal between 0 and 1 (i.e., the similarity score of two different style images is generally not 0 or 1), with a larger value indicating greater similarity. The specific methods for obtaining the similarity score may include, but are not limited to, obtaining it based on statistical averages or using a random forest voting mechanism in machine learning, etc. Specifically, the similarity score between the standard display style of the goods and promotional materials and the display style to be inspected is obtained by comparing the promotional material features, overall mask image features, overall segmentation image features, and high-frequency product features of the standard display style of the goods and promotional materials with those of the display style to be inspected, including but not limited to steps S71 to S75.

[0082] S71. Compare the standard display style of the goods and promotional materials with the characteristics of the promotional materials in the display style to be inspected, and obtain the similarity S of the promotional materials expressed by the following formula. p :

[0083]

[0084] In the formula, A p The promotional material characteristics, B, represent the standard display style of the aforementioned merchandise and promotional materials. pThe promotional material feature representing the merchandise and promotional material to-be-inspected display style, CS() represents a cosine distance function.

[0085] In the step S71, since only A p means that there is promotional material in the merchandise and promotional material standard display style, but no promotional material in the merchandise and promotional material to-be-inspected display style, therefore the promotional material similarity S p will be zero, so as to represent that there is obvious difference in the promotional material dimension. And since only B p means that there is no promotional material in the merchandise and promotional material standard display style, but there is promotional material in the merchandise and promotional material to-be-inspected display style, therefore the promotional material similarity S p will also be zero, so as to represent that there is obvious difference in the promotional material dimension.

[0086] S72. Comparing the overall mask pattern feature of the merchandise and promotional material standard display style and the merchandise and promotional material to-be-inspected display style, obtaining an overall mask pattern similarity S1 represented by the following formula:

[0087] S1 = CS(A1, B1)

[0088] In the formula, A1 represents the overall mask pattern feature of the merchandise and promotional material standard display style, and B1 represents the overall mask pattern feature of the merchandise and promotional material to-be-inspected display style.

[0089] S73. Comparing the overall segmentation pattern feature of the merchandise and promotional material standard display style and the merchandise and promotional material to-be-inspected display style, obtaining an overall segmentation pattern similarity S2 represented by the following formula:

[0090] S2 = CS(A2, B2)

[0091] In the formula, A2 represents the overall segmentation pattern feature of the merchandise and promotional material standard display style, and B1 represents the overall segmentation pattern feature of the merchandise and promotional material to-be-inspected display style.

[0092] S74. Comparing the high-frequency merchandise feature of the merchandise and promotional material standard display style and the merchandise and promotional material to-be-inspected display style, obtaining a high-frequency merchandise similarity S m .

[0093] In the step S74, the high-frequency merchandise similarity S m may be calculated by referring to the Bleu score (which is an evaluation index in the field of text translation technology; in the embodiment, the index can be borrowed to measure the similarity of two high-frequency merchandise queues) or the recall rate and other calculation formula.

[0094] S75. The similarity S between the promotional material of the standard display style and the promotional material of the to-be-inspected display style is calculated according to the promotional material similarity S p , the overall mask map similarity S1, the overall segmentation map similarity S2 and the high-frequency commodity similarity S m , according to the following formula:

[0095] S = IF(S p > S p,thres ) x ((η1 x S1 + η2 x S2) + β x S m )

[0096] In the formula, S p,thres represents a preset promotional material similarity threshold, η1 and η2 respectively represent weight coefficients and have η1 + η2 = 1, β represents a preset penalty coefficient, and IF() represents an indicator function.

[0097] S8. It is judged whether the similarity is less than a preset similarity threshold. If yes, it is determined that the inspection result of the to-be-inspected display style of the commodity and the promotional material is unqualified. Otherwise, it is determined that the inspection result of the to-be-inspected display style of the commodity and the promotional material is qualified.

[0098] The commodity and promotional material display style inspection method described in the foregoing steps S1-S8 provides a new scheme for inspecting the display style of commodities and promotional materials based on computer vision technology, that is, first, the feature extraction processing is performed on the style image of the standard display style of commodities and promotional materials to obtain the promotional material feature, the overall mask map feature, the overall segmentation map feature and the high-frequency commodity feature of the standard display style. Then, the promotional material target detection processing and the commodity semantic segmentation processing are performed on the style image of the to-be-inspected display style of commodities and promotional materials to obtain the promotional material feature, the overall mask map feature, the overall segmentation map feature and the high-frequency commodity feature of the to-be-inspected display style. Then, the multi-dimensional features of the standard display style and the to-be-inspected display style are compared to obtain the similarity of the two styles. Finally, according to the comparison result of the similarity and the threshold, the inspection result of the to-be-inspected display style is determined. In this way, it is not necessary to send a store inspection supervisor to the site of a chain store for offline inspection, so as to reduce the labor cost, avoid human errors and improve the inspection efficiency, which is convenient for practical application and promotion. In addition, in view of the problem that there are many drugs and medical devices in the medical retail field and the packaging is very similar, through the unsupervised pre-training, semi-supervised fine-tuning and distillation of the embodiment, the recognition and feature extraction effect of drugs, medical devices, drug and medical device combinations and commodity poster single products can be improved. Since the features / texture structures of the style image, the mask image and the sub-image are combined to integrate different perspective information to calculate the style similarity, the fairness of the inspection result is guaranteed, and the embodiment is particularly suitable for the medical retail field.

[0099] As Figure 3 shown in the second aspect of the present embodiment provides a virtual device for implementing the merchandise and promotional material display style checking method of the first aspect, comprising a style image acquisition unit, a standard feature extraction unit, a target detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit and a checking result determination unit;

[0100] The style image acquisition unit is configured to acquire a style image of a standard display style of merchandise and promotional materials, wherein the standard display style of merchandise and promotional materials refers to a standard display style of merchandise and / or promotional materials.

[0101] The standard feature extraction unit is in communication connection with the style image acquisition unit and is configured to perform merchandise and promotional material feature extraction processing on the style image of the standard display style of merchandise and promotional materials, to obtain promotional material features, overall mask feature, overall segmentation feature and high-frequency merchandise features of the standard display style of merchandise and promotional materials, wherein the high-frequency merchandise features include the top N merchandise arranged in order from high to low according to the frequency of occurrence, and N represents a positive integer greater than or equal to 2.

[0102] The style image acquisition unit is further configured to acquire a style image of a display style to be checked of merchandise and promotional materials, wherein the display style to be checked of merchandise and promotional materials refers to a display style to be checked of merchandise and / or promotional materials.

[0103] The target detection processing unit is in communication connection with the style image acquisition unit and is configured to perform promotional material target detection processing on the style image of the display style to be checked of merchandise and promotional materials, to obtain sub-images of each promotional material in the display style to be checked of merchandise and promotional materials, and to perform feature extraction processing on the sub-images of each promotional material, to obtain the promotional material features of the display style to be checked of merchandise and promotional materials.

[0104] The semantic segmentation processing unit is in communication connection with the style image acquisition unit and is configured to perform merchandise semantic segmentation processing on the style image of the display style to be checked of merchandise and promotional materials, to obtain semantic segmentation results of different merchandise in the display style to be checked of merchandise and promotional materials, and to count the frequency of occurrence of the corresponding merchandise in the display style to be checked of merchandise and promotional materials according to the semantic segmentation results, to obtain the high-frequency merchandise features of the display style to be checked of merchandise and promotional materials.

[0105] The image integration processing unit is respectively connected in communication with the target detection processing unit and the semantic segmentation processing unit, and is configured to integrate the sub-image of each promotional material and the semantic segmentation result of different commodities, to obtain an overall mask image and an overall segmentation image of the to-be-inspected display style of the commodities and promotional materials, to perform feature extraction processing on the overall mask image of the to-be-inspected display style of the commodities and promotional materials, to obtain the overall mask image feature of the to-be-inspected display style of the commodities and promotional materials, and to perform feature extraction processing on the overall segmentation image of the to-be-inspected display style of the commodities and promotional materials, to obtain the overall segmentation image feature of the to-be-inspected display style of the commodities and promotional materials.

[0106] The style similarity determination unit is respectively connected in communication with the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit and the image integration processing unit, and is configured to respectively and one-to-one compare the promotional material features, the overall mask image features, the overall segmentation image features and the high-frequency commodity features of the standard display style of the commodities and promotional materials and the to-be-inspected display style of the commodities and promotional materials, to obtain the similarity of the standard display style of the commodities and promotional materials and the to-be-inspected display style of the commodities and promotional materials.

[0107] The inspection result determination unit is configured to determine whether the similarity is less than a preset similarity threshold value, and if yes, to determine that the inspection result of the to-be-inspected display style of the commodities and promotional materials is unqualified, and otherwise to determine that the inspection result of the to-be-inspected display style of the commodities and promotional materials is qualified.

[0108] The working process, working details and technical effects of the foregoing device provided by the second aspect of the embodiment can be referred to the commodity and promotional material display style inspection method described in the first aspect, which will not be described here again.

[0109] As Figure 4As shown, the third aspect of the present embodiment provides a computer device for performing the merchandise and promotional material display style checking method according to the first aspect, which comprises a memory, a processor and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to transmit and receive messages, and the processor is configured to read the computer program and perform the merchandise and promotional material display style checking method according to the first aspect. Specifically, the memory can include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in first-out memory (FIFO) and / or a first-in last-out memory (FILO), etc.; and the processor can be, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device can further include, but is not limited to, a power module, a display screen and other necessary components.

[0110] The working process, working details and technical effects of the aforementioned computer device provided by the third aspect of the present embodiment can be referred to the merchandise and promotional material display style checking method according to the first aspect, which will not be described here again.

[0111] The fourth aspect of the present embodiment provides a computer readable storage medium storing instructions of the merchandise and promotional material display style checking method according to the first aspect, i.e., the computer readable storage medium stores instructions, and when the instructions are run on a computer, the merchandise and promotional material display style checking method according to the first aspect is performed. The computer readable storage medium refers to a carrier storing data, which can include, but is not limited to, floppy disks, optical disks, hard disks, flash memories, USB flash drives and / or memory sticks, etc., and the computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0112] The working process, working details and technical effects of the aforementioned computer readable storage medium provided by the fourth aspect of the present embodiment can be referred to the merchandise and promotional material display style checking method according to the first aspect, which will not be described here again.

[0113] The fifth aspect of the present embodiment provides a computer program product comprising a computer program or instructions, which, when executed by a computer, implement the merchandise and promotional material display style checking method according to the first aspect. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0114] Finally, it should be noted that the above description is only the preferred embodiment of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for checking the display style of goods and promotional materials, characterized in that, include: Obtain a standard display image of merchandise and promotional materials, wherein the standard display style of merchandise and promotional materials refers to the standard display style of merchandise and / or promotional materials; The product and promotional material standard display style image is processed by extracting product and promotional material features to obtain promotional material features, overall mask image features, overall segmentation image features and high-frequency product features of the product and promotional material standard display style. The high-frequency product features include the top N products arranged in descending order of frequency of occurrence, where N represents a positive integer greater than or equal to 2. Obtain the image of the display pattern of the goods and promotional materials to be inspected, wherein the display pattern of the goods and promotional materials to be inspected refers to the display pattern of the goods and / or promotional materials to be inspected; The image of the product and promotional material display style to be inspected is processed by the target detection of promotional materials to obtain sub-images of each promotional material in the product and promotional material display style to be inspected, and the feature extraction processing of each sub-image of promotional materials is performed to obtain the promotional material features of the product and promotional material display style to be inspected. The process involves performing semantic segmentation on the image of the product and promotional material display pattern to obtain semantic segmentation results for different products within the display pattern. Based on these semantic segmentation results, the frequency of occurrence of each product within the display pattern is calculated to obtain the high-frequency product features. Specifically, this includes: for different products, based on the semantic segmentation results containing at least one masked connected region, using the number of corresponding masked connected regions or the total pixel area as the corresponding frequency of occurrence within the display pattern; and then, based on the product ranking results according to the frequency of occurrence, obtaining the high-frequency product features of the display pattern. By integrating the sub-images of each promotional material and the semantic segmentation results of different products, an overall mask image and an overall segmentation image of the product and promotional material display style to be inspected are obtained. Feature extraction processing is performed on the overall mask image of the product and promotional material display style to be inspected to obtain the overall mask image features of the product and promotional material display style to be inspected. Feature extraction processing is also performed on the overall segmentation image of the product and promotional material display style to be inspected to obtain the overall segmentation image features of the product and promotional material display style to be inspected. The similarity between the standard display style of the goods and promotional materials and the display style to be inspected is obtained by comparing the promotional material features, overall mask image features, overall segmentation image features, and high-frequency product features of the standard display style of the goods and promotional materials with those of the display style to be inspected, one-to-one. Specifically, this includes comparing the promotional material features of the standard display style of the goods and promotional materials with those of the display style to be inspected, and obtaining the promotional material similarity S expressed by the following formula. p : In the formula, A p The promotional material characteristics, B, represent the standard display style of the aforementioned merchandise and promotional materials. p The promotional material features represent the display style of the goods and promotional materials to be inspected, and CS() represents the cosine distance function; the overall mask image features of the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected are compared to obtain the overall mask image similarity S1 expressed by the following formula: S1 = CS(A1, B1) In the formula, A1 represents the overall mask image feature of the standard display style of the goods and promotional materials, and B1 represents the overall mask image feature of the display style of the goods and promotional materials to be inspected; by comparing the overall segmentation image features of the standard display style of the goods and promotional materials with those of the display style of the goods and promotional materials to be inspected, the overall segmentation image similarity S2 is obtained as expressed by the following formula: S2 = CS(A2, B2) In the formula, A2 represents the overall segmentation feature of the standard display style of the goods and promotional materials, and B1 represents the overall segmentation feature of the display style of the goods and promotional materials to be inspected; by comparing the high-frequency product features of the standard display style of the goods and promotional materials with those of the display style of the goods and promotional materials to be inspected, the high-frequency product similarity S is obtained. m Based on the similarity S of the promotional materials p The overall mask image similarity S1, the overall segmentation image similarity S2, and the high-frequency product similarity S... m The similarity S between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is calculated according to the following formula: S=I(S p >S p,thres )×((η1×S1+η2×S2)+β×S m ) In the formula, S p,thres η1 and η2 represent the preset similarity threshold of promotional materials, respectively, and η1 + η2 = 1. β represents the preset coefficient, and I() represents the indicator function. If the similarity is less than a preset similarity threshold, the inspection result of the product and promotional material display style to be inspected is determined to be unqualified; otherwise, the inspection result of the product and promotional material display style to be inspected is determined to be qualified.

2. The method for checking the display style of goods and promotional materials according to claim 1, characterized in that, Obtain sample images of standard display styles for merchandise and promotional materials, including: A first image is captured by a mobile camera when the standard display style of the goods and promotional materials is within a preset area of ​​interest in the field of view of the lens, and the first image is used as the style image of the standard display style of the goods and promotional materials, wherein the standard display style of the goods and promotional materials refers to the standard display style of the goods and / or promotional materials. Alternatively, first, a second image is obtained by the camera capturing the standard display style of the goods and promotional materials. Then, in response to the human-computer interaction operation, the annotation result of marking the region of interest of the standard display style of the goods and promotional materials in the second image is obtained. Finally, the style image of the standard display style of the goods and promotional materials is extracted from the second image according to the annotation result. The standard display style of the goods and promotional materials refers to the standard display style of the goods and / or promotional materials. Alternatively, a third image can be obtained by a camera capturing the standard display style of the goods and promotional materials at a preset point of interest, and the third image can be used as the style image of the standard display style of the goods and promotional materials. The point of interest refers to the camera position that can capture the information of interest of the standard display style of the goods and promotional materials, and the standard display style of the goods and promotional materials refers to the standard display style of the goods and / or promotional materials.

3. The method for checking the display style of goods and promotional materials according to claim 1, characterized in that, The image of the standard display style of the goods and promotional materials is processed by extracting the features of the goods and promotional materials to obtain the promotional material features, overall mask image features, overall segmentation image features, and high-frequency product features of the standard display style of the goods and promotional materials, including: The style image of the standard display style of the goods and promotional materials is imported into the product and promotional material feature extraction model pre-trained based on neural network and unsupervised learning. The output is the promotional material features, overall mask image features, overall segmentation image features and high-frequency product features of the standard display style of the goods and promotional materials. The high-frequency product features include the top N products arranged in descending order of frequency of occurrence, where N represents a positive integer greater than or equal to 2.

4. The method for checking the display style of goods and promotional materials according to claim 1, characterized in that, The image of the product and promotional material display to be inspected is processed by target detection of promotional materials to obtain sub-images of each promotional material in the product and promotional material display to be inspected, including: The style image of the product and promotional material display to be inspected is imported into the promotional material recognition model pre-trained based on the YOLO object detection algorithm, and the sub-images of each promotional material in the product and promotional material display to be inspected are output.

5. The method for checking the display style of goods and promotional materials according to claim 1, characterized in that, The product semantic segmentation process is performed on the style image of the product and promotional material display style to be inspected, and the semantic segmentation results of different products in the product and promotional material display style to be inspected are obtained, including: The style image of the product and promotional material display to be inspected is imported into the product semantic segmentation model pre-trained based on edge detection network and semantic segmentation network to obtain the semantic segmentation results of different products in the product and promotional material display to be inspected.

6. A device for checking the display style of goods and promotional materials, characterized in that, It includes a style image acquisition unit, a standard feature extraction unit, an object detection processing unit, a semantic segmentation processing unit, an image integration processing unit, a style similarity determination unit, and a check result judgment unit; The style image acquisition unit is used to acquire style images of standard display styles of goods and promotional materials, wherein the standard display style of goods and promotional materials refers to the standard display style of goods and / or promotional materials. The standard feature extraction unit is communicatively connected to the style image acquisition unit and is used to perform product and promotional material feature extraction processing on the style image of the standard display style of the product and promotional materials to obtain the promotional material features, overall mask image features, overall segmentation image features and high-frequency product features of the standard display style of the product and promotional materials. The high-frequency product features include the top N products arranged in descending order of frequency of occurrence, where N represents a positive integer greater than or equal to 2. The style image acquisition unit is also used to acquire style images of the display styles of goods and promotional materials to be inspected, wherein the display styles of goods and promotional materials to be inspected refer to the display styles of goods and / or promotional materials to be inspected. The target detection processing unit is communicatively connected to the style image acquisition unit, and is used to perform promotional material target detection processing on the style image of the product and promotional material display style to be inspected, to obtain sub-images of each promotional material in the product and promotional material display style to be inspected, and to perform feature extraction processing on the sub-images of each promotional material to obtain the promotional material features of the product and promotional material display style to be inspected. The semantic segmentation processing unit, communicatively connected to the style image acquisition unit, is used to perform semantic segmentation processing on the style image of the product and promotional material display style to be inspected, to obtain the semantic segmentation results of different products in the product and promotional material display style to be inspected, and to count the frequency of occurrence of the corresponding products in the product and promotional material display style to be inspected based on the semantic segmentation results, thereby obtaining the high-frequency product features of the product and promotional material display style to be inspected. Specifically, this includes: for different products, based on the corresponding semantic segmentation results that contain at least one masked connected region, taking the number of corresponding masked connected regions or the total pixel area as the corresponding frequency of occurrence in the product and promotional material display style to be inspected; and then obtaining the high-frequency product features of the product and promotional material display style to be inspected based on the product ranking results of the frequency of occurrence. The image integration processing unit is communicatively connected to the target detection processing unit and the semantic segmentation processing unit, respectively. It is used to integrate the sub-images of each promotional material and the semantic segmentation results of different products to obtain the overall mask image and overall segmentation image of the product and promotional material display style to be inspected. It also performs feature extraction processing on the overall mask image of the product and promotional material display style to be inspected to obtain the overall mask image features of the product and promotional material display style to be inspected, and performs feature extraction processing on the overall segmentation image of the product and promotional material display style to be inspected to obtain the overall segmentation image features of the product and promotional material display style to be inspected. The style similarity determination unit is communicatively connected to the standard feature extraction unit, the target detection processing unit, the semantic segmentation processing unit, and the image integration processing unit. It is used to compare the standard display style of the goods and promotional materials with the display style to be inspected, specifically comparing the promotional material features, overall mask image features, overall segmentation image features, and high-frequency product features to obtain the similarity between the standard display style and the display style to be inspected. Specifically, this includes comparing the promotional material features of the standard display style and the display style to be inspected to obtain the promotional material similarity S expressed by the following formula. p : In the formula, A p The promotional material characteristics, B, represent the standard display style of the aforementioned merchandise and promotional materials. p The promotional material features represent the display style of the goods and promotional materials to be inspected, and CS() represents the cosine distance function; the overall mask image features of the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected are compared to obtain the overall mask image similarity S1 expressed by the following formula: S1 = CS(A1, B1) In the formula, A1 represents the overall mask image feature of the standard display style of the goods and promotional materials, and B1 represents the overall mask image feature of the display style of the goods and promotional materials to be inspected; by comparing the overall segmentation image features of the standard display style of the goods and promotional materials with those of the display style of the goods and promotional materials to be inspected, the overall segmentation image similarity S2 is obtained as expressed by the following formula: S2 = CS(A2, B2) In the formula, A2 represents the overall segmentation feature of the standard display style of the goods and promotional materials, and B1 represents the overall segmentation feature of the display style of the goods and promotional materials to be inspected; by comparing the high-frequency product features of the standard display style of the goods and promotional materials with those of the display style of the goods and promotional materials to be inspected, the high-frequency product similarity S is obtained. m Based on the similarity S of the promotional materials p The overall mask image similarity S1, the overall segmentation image similarity S2, and the high-frequency product similarity S... m The similarity S between the standard display style of the goods and promotional materials and the display style of the goods and promotional materials to be inspected is calculated according to the following formula: S=I(S p >S p,thres )×((η1×S1+η2×S2)+β×S m ) In the formula, S p,thres η1 and η2 represent the preset similarity threshold of promotional materials, respectively, and η1 + η2 = 1. β represents the preset coefficient, and I() represents the indicator function. If the similarity is less than a preset similarity threshold, the inspection result of the product and promotional material display style to be inspected is determined to be unqualified; otherwise, the inspection result of the product and promotional material display style to be inspected is determined to be qualified. The inspection result determination unit is used to determine whether the similarity is less than a preset similarity threshold. If so, the inspection result of the product and promotional material display style to be inspected is determined to be unqualified; otherwise, the inspection result of the product and promotional material display style to be inspected is determined to be qualified.

7. A computer device, characterized in that, The device includes a memory, a processor, and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the product and promotional material display style verification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that... The computer-readable storage medium stores instructions that, when executed on a computer, perform the product and promotional material display style verification method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the computer, they implement the method for checking the display style of goods and promotional materials as described in any one of claims 1 to 5.

Images