Artificial intelligence-based display item review method and system

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses an artificial intelligence-based display item auditing method and system, comprising: utilizing a first standard of display rules to audit items at each display location; if the audit result is not reasonable for the display, auditing the displayed items using a second standard based on the audit result of the first standard; generating a joint audit result based on the results of the two audits; and generating an adjustment strategy if the joint audit result is unreasonable for the display. This system achieves comprehensive judgment capabilities from the overall to the local, from rule matching to semantic understanding. It can effectively find a display adjustment plan with minimal intervention, improving the accuracy and operability of the adjustment strategy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based display item review method and system. Background Art

[0002] In retail, trade shows, supermarkets, and other real-world scenarios, the way merchandise is displayed not only influences consumers' visual perception and purchasing desire but also directly impacts brand promotion effectiveness and sales efficiency. Therefore, effective, reasonable, and standardized review of merchandise display has become a key issue in intelligent and automated retail management.

[0003] Currently, traditional display audits rely primarily on manual inspections or simple rule template comparisons, resulting in numerous issues such as low audit efficiency, high subjectivity, inconsistent standards, and an inability to adapt to diverse scenarios. With the development of computer vision and artificial intelligence technologies, some companies have attempted to introduce image recognition models for product detection and identification. However, existing technologies generally remain at the target recognition level, lacking in-depth semantic understanding and structural rule analysis of "display rationality," making it difficult to meet the business needs of precise audits.

[0004] Especially when faced with complex display rules such as brand characteristics, layout logic, and hierarchical structures, existing solutions based on static template comparisons struggle to support dynamic adaptation and strategic feedback, nor can they generate optimal adjustment plans based on changing scenarios. More importantly, they lack a coordinated assessment of the relationship between local rationality and global structure, resulting in limited review results and a failure to form overall optimization guidance. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an artificial intelligence-based display item review method, comprising:

[0007] Obtain an image of items displayed in a display area and identify the display location of each item based on the image content;

[0008] Using the first standard of the display rules, the items on each display position are reviewed; if the review result is that the display is reasonable, the review result is directly output; if the review result is not reasonable, the displayed items are reviewed according to the second standard based on the review result of the first standard;

[0009] Generate a joint audit result based on the results of the two audits; if the result of the joint audit is that the display is reasonable, directly output the joint audit result; if the result of the joint audit is that the display is unreasonable, generate an adjustment strategy;

[0010] The adjustment strategy includes adjusting the display of items so that the result of the joint audit is that the display is reasonable, and outputting a solution with the shortest adjustment path.

[0011] As a preferred embodiment of the artificial intelligence-based display item auditing method of the present invention, the image content recognition result includes the identification of the display item location, display item defects, and display item type through a trained object detection model;

[0012] The recognition process of the target detection model is divided into two stages: in the first stage, the position and type of displayed items are directly identified using the target detection model;

[0013] In the second stage, the types of displayed items at each position are used to match the corresponding attention weights through a preset mapping relationship; attention is allocated to the target detection model based on the attention weights; and the target detection model after attention allocation is used to identify defects in the displayed items at each position.

[0014] As a preferred solution of the artificial intelligence-based display item audit method described in the present invention, after the identification is completed, the defects of each display item are recorded, and the defect identification results are output at the same time as the audit results.

[0015] As a preferred embodiment of the artificial intelligence-based display item review method of the present invention, the first criterion includes reviewing the display according to the consistency principle of pre-selected features;

[0016] Based on the features pre-selected during human-computer interaction before the review, if the pre-selected features are non-numerical features, the non-numerical pre-selected features will be quantified and the review will be conducted based on the quantified pre-selected features;

[0017] For each row in the display area, feature consistency is reviewed: for each row, the maximum and minimum feature values ​​are screened to ensure that the minimum feature value of the previous row is greater than or equal to the maximum feature value of the current row in one direction or from the middle to both sides; and within each row, the feature values ​​of the items decrease in one direction or from the middle to both sides.

[0018] When the audit of the consistency of the characteristics is met, the audit result is judged to be reasonable display.

[0019] As a preferred embodiment of the artificial intelligence-based display item review method of the present invention, the second standard includes inheriting the preselected features in the first standard to review the rationality of the display; wherein the rationality is categorized into: regional rationality and supplementary rationality;

[0020] The supplementary rationality is achieved through quantitative analysis. When the pre-selected features are non-numerical features, in the nth row, if the number of remaining display positions after the items with feature i are displayed is equal to the number of items with feature j, then the nth row is judged to meet the rationality; otherwise, the nth row is judged to not meet the rationality.

[0021] Among them, i and j represent different feature indexes in the same feature;

[0022] The regional rationality is achieved by segmenting the display area: within the display area, the display area is divided into sub-areas to obtain sub-area combinations, so that the display of items in each sub-area satisfies the feature consistency review, or satisfies the combined review of the feature consistency and the supplementary rationality; the sub-area combination with the least sub-areas is output as the segmentation result of the display area;

[0023] Through the pre-trained neural network, the rationality of the segmentation results of the display area is analyzed, and a binary classification judgment is obtained to determine whether the regional segmentation results are reasonable.

[0024] As a preferred embodiment of the artificial intelligence-based display item audit method of the present invention, the joint audit includes, firstly based on the regional rationality, conducting a comprehensive audit of the feature consistency and the supplementary rationality; if the result of the comprehensive audit is that the display is unreasonable, then judging the regional rationality;

[0025] If the judgment result of the regional rationality is that the regional segmentation result is reasonable, the result of the joint review is output as qualified display; otherwise, the result of the joint review is output as unqualified display;

[0026] The comprehensive audit includes, in the process of judging regional rationality, excluding all rows judged to be reasonable from the display area. If the audit result of the feature consistency in the display area after elimination is that the display is reasonable, then the result of the joint audit is directly output as qualified display; otherwise, the result of the comprehensive audit is generated as unreasonable display.

[0027] As a preferred solution of the artificial intelligence-based display item audit method of the present invention, the adjustment strategy further includes generating the strategy using an optimization algorithm based on energy conservation;

[0028] The optimization algorithm based on energy conservation includes assuming that each displayed item is an indivisible energy; in the display area, by rearranging the energy, the whole area is kept in a steady state;

[0029] If the display of items is judged to be reasonable after review, it means that the overall situation is in a stable state;

[0030] In the display area, items are removed at any position. After removing D items, if the global state is stable, the positions and item marks of D items are recorded. By testing different removal strategies, the position and mark of the corresponding items are selected when D is the minimum value.

[0031] Recorded as: minimum number of eliminations , the corresponding location set of displayed items , corresponding to the mark of the displayed items ;

[0032] in, Indicates the display position of the rth elimination, Indicates the mark of the rth item to be removed; Indicates the number of items removed;

[0033] The specific steps of the rearrangement are:

[0034] Step 1: Assign the remaining display positions to W and get , as an arrangable position;

[0035] Step 2: Make the elements in Q Randomly arrange the data, and then analyze whether the global data is in a steady state;

[0036] Step 3: If there is a steady-state arrangement, calculate the sum of the distances L before and after adjustment of each item, and output the arrangement with the smallest L. If there is no steady-state arrangement, randomly select one of the displayed items that have not been eliminated as the eliminated display item, repeat the analysis of steps 1 and 2, and after going through all the eliminated display items, output the arrangement with the smallest L. If no output can be obtained by randomly selecting one of the displayed items that have not been eliminated, increase the number of randomly selected items until the output is obtained.

[0037] An artificial intelligence-based display item review system, wherein:

[0038] The acquisition unit acquires images of items displayed in the display area and identifies the display position of each item based on the image content;

[0039] The review unit reviews the items at each display location using the first standard of the display rules; if the review result shows that the display is reasonable, the review result is directly output; if the review result shows that the display is not reasonable, the displayed items are reviewed using the second standard based on the review result of the first standard;

[0040] The analysis unit generates a joint audit result based on the results of the two audits; if the result of the joint audit is that the display is reasonable, the analysis unit directly outputs the joint audit result; if the result of the joint audit is that the display is unreasonable, the analysis unit generates an adjustment strategy.

[0041] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0042] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0043] Beneficial effects of the present invention: The artificial intelligence-based display item audit method provided by the present invention realizes multi-dimensional intelligent audit of display items by introducing multi-stage target detection, pre-selected feature consistency audit mechanism, and regional and supplementary rationality judgment strategies. Through structural analysis based on image recognition, combined with the audit features set by human-computer interaction, a comprehensive judgment capability from the whole to the part, from rule matching to semantic understanding is achieved. In particular, the present invention adopts an optimization algorithm based on the idea of ​​energy conservation, which can effectively find the display adjustment plan with minimum intervention and improve the accuracy and operability of the adjustment strategy. Compared with the existing technology, the present invention has significant improvements in audit accuracy, adaptability to scene diversity and automatic adjustment capabilities. It is suitable for many scenarios such as large supermarkets, chain brands, smart stores, etc., and has high practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 An overall flow chart of an artificial intelligence-based display item review method provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0047] Reference Figure 1 , as one embodiment of the present invention, provides an artificial intelligence-based display item review method, comprising:

[0048] S1: Acquire an image of items displayed in a display area, and identify the display position of each item based on the image content.

[0049] Furthermore, the trained object detection model is used to identify the location, defects, and types of displayed items. The object detection model's recognition process is divided into two stages: In the first stage, the object detection model is used to directly identify the location and type of displayed items.

[0050] In the second stage, the types of displayed items at each position are used to match the corresponding attention weights through a preset mapping relationship (this weight is preset and is generally weighted for the positions where defects of a certain item are prone to occur); attention is allocated to the target detection model based on the attention weights; and the target detection model after attention allocation is used to identify defects in the displayed items at each position.

[0051] A two-stage recognition structure is introduced: the first stage focuses on macro-recognition, quickly locking the location and type information of each object and establishing a basic structural map; the second stage introduces the semantic association between type and defect, and through a preset mapping mechanism, matches the identified object type with the corresponding attention weight to guide the target detection model to focus on defect-prone areas, thereby improving the accuracy and robustness of defect detection.

[0052] The model's focus area can be adaptively adjusted based on the characteristics of different item types, achieving category-driven attention allocation and depth perception optimization, effectively improving defect recognition performance in complex display environments. Furthermore, phased recognition enables dynamic scheduling of model resources, improving recognition efficiency and adapting to the high-concurrency, high-precision auditing requirements of actual retail scenarios.

[0053] In this embodiment, the object detection model is preferably the YOLOv5 (You Only Look Once version 5) model, built on a deep convolutional neural network. This model provides end-to-end object localization and classification capabilities, enabling high-precision detection of multiple merchandise objects in display images while ensuring real-time performance. The YOLOv5 model utilizes a CSPDarknet backbone network for feature extraction and incorporates a PANet architecture for cross-layer feature fusion. This effectively identifies object location bounding boxes, category labels, and confidence values, meeting the recognition requirements for multiple, densely packed objects in retail scenarios.

[0054] During the first stage of recognition, the YOLOv5 model is used to obtain the bounding box location and category identifier for each item, providing input for attention guidance in the second stage. This model has been trained on a display dataset containing images of multiple product categories and demonstrates strong adaptability to various scenarios.

[0055] In an optional embodiment, the target detection model may also include but is not limited to: YOLOv8, Faster R-CNN, DETR and other models.

[0056] After the identification is completed, the defects of each displayed item are recorded, and the defect identification results are output at the same time as the audit results.

[0057] S2: Using the first standard of the display rules, review the items at each display location; if the review result is that the display is reasonable, directly output the review result; if the review result is not that the display is reasonable, then based on the review result of the first standard, review the displayed items according to the second standard.

[0058] The first criterion consists in reviewing the display in accordance with the principle of consistency of pre-selected characteristics.

[0059] Based on the pre-selected features during human-computer interaction before the review (for example, the volume, height, brand, year, or purpose of the item are preset as standards. Multiple features can be set during the preset process), if the pre-selected features are non-numerical features, the non-numerical pre-selected features are quantified and the review is conducted based on the quantified pre-selected features (for example, if the preset feature is color, the color features are quantified before analysis).

[0060] For each row in the display area, the feature consistency is audited: the maximum and minimum values ​​of the features of each row are screened, so that the minimum value of the feature of the previous row in one direction or from the middle position to both sides is greater than or equal to the maximum value of the feature of the current row. And in each row, the feature value of the items decreases in one direction or from the middle position to both sides. In fact, it can be simply understood as: when auditing the consistency of display features, for each row in the display area, based on the pre-selected feature set , extract the corresponding feature value set of all items in this row ,in Indicates the current behavior Line; define The maximum and minimum values ​​of row features are: and .

[0061] If there are two rows The following decreasing relationship is satisfied: . It is considered that the current two rows meet the judgment condition of "hierarchical feature decreasing consistency".

[0062] If the audit mode is set to "Decrease symmetrically from the center to both sides", the current behavior is set The row and column index range is ;like ,but Anyway, ; That is, the items decrease from the middle column to the left and right sides, forming a symmetrical characteristic distribution.

[0063] By applying the principle of consistency based on pre-selected features, we construct a basic display review rule system that is both universal and structured. In real-world retail, exhibition, and warehousing scenarios, item displays require a distinct sense of hierarchy, symmetry, and categorization. This design uses human-computer interaction to pre-select review features, such as brand, volume, color, and purpose, enabling personalized and adaptive review logic configuration.

[0064] To support a wider range of audit feature adaptations, this solution specifically introduces a quantification mechanism for non-numerical features, such as encoding colors into HSV values ​​and mapping brands into grade numbers, so that unstructured information can be converted into a unified numerical scale, facilitating algorithm processing and consistency calculation.

[0065] This design further introduces a decreasing consistency rule across rows at the structural level, requiring that the minimum eigenvalue of the previous row should not be less than the maximum value of the current row, strengthening the hierarchical progressive logic of the display. At the same time, the "center-symmetrical decreasing" or "one-way decreasing" layout rules are introduced in the horizontal structure, so that the system has a stronger symmetry recognition ability when detecting the internal structure of each row. This audit mechanism is not only applicable to numerical features with natural order relationships such as height and size, but also to feature values ​​with "soft sorting" logic such as color gradients and category aggregation, which greatly improves the applicability and fault tolerance of the audit algorithm. When the audit of the consistency of the above features is met, the audit result is judged to be reasonable display.

[0066] The second standard includes inheriting the pre-selected features in the first standard and conducting a rationality review of the display; wherein the rationality is classified into: regional rationality and supplementary rationality.

[0067] The rationality of the supplementary approach is achieved through quantitative analysis. When the pre-selected features are non-numerical, in row n, if the number of remaining display positions after the items with feature i are displayed is equal to the number of items with feature j, then row n is judged to meet rationality; otherwise, row n is judged to not meet rationality. This judgment refers to "small numbers of items" that "fill gaps" in the display. If there are too many such items, it means that the supplementary items have actually formed a scale and are not suitable as supplementary display positions for items with a small base number. Here, i and j represent different feature indexes within the same feature.

[0068] Traditional consistency audit methods tend to judge the arrangement logic and uniformity of overall features. However, for individual categories that are rare in number but still need to be displayed in stores, if they are forced to be constrained by unified rules, it may lead to audit misjudgments or even reasonable displays being mistakenly identified as unqualified.

[0069] To this end, a supplementary rationality judgment mechanism is proposed. Using preselected features as a reference, it quantifies and encodes non-numerical features, and then performs a structured analysis of the feature distribution of items within a row. Specifically, if, after a certain type of characteristic item (denoted as characteristic i) completes the primary display, the number of remaining slots in the current row is exactly equal to the number of another type of small-cardinality characteristic (characteristic j), then characteristic j is considered a reasonable supplementary item, and its presence constitutes a natural extension of the primary display logic. This judgment logic reflects the design principle of "small quantity, filling, and non-dominant structure." It aims to achieve automatic judgment through structural analysis and quantity matching, identifying supplementary items that do not visually or structurally disrupt the primary logical display order, thereby enhancing the system's flexible adaptability to complex real-world scenarios.

[0070] Conversely, if the number of items with characteristic j exceeds the remaining available slots, forming a primary structural position, this indicates that the category no longer possesses the "supplementary attribute" and should not be considered a reasonable replacement, resulting in the row being judged as irrational. This mechanism balances the proportional relationship between characteristic dimensions with structural rationality, effectively improving the inclusiveness and intelligence of the review strategy. It is particularly suitable for complex layouts such as non-standard structures, temporary displays, and combined sales scenarios.

[0071] Regional rationality is achieved by segmenting the display area: within the display area, the display area is divided into sub-areas, resulting in sub-area combinations such that the item display in each sub-area meets the feature consistency review or meets a combination of feature consistency and supplementary rationality (this condition refers to either: 1. Each area meets consistency; 2. Consistency supplemented by "supplementary rationality"). The sub-area combination with the fewest sub-areas is output as the display area segmentation result. A pre-trained neural network is used to analyze the rationality of the display area segmentation results, resulting in a binary classification judgment on whether the regional segmentation results are reasonable.

[0072] S3: Generate the results of the joint audit based on the results of the two audits.

[0073] Specifically, the method includes: prioritizing the regional rationality, comprehensively reviewing the feature consistency and the supplementary rationality; if the comprehensive review results in an unreasonable display, then judging the regional rationality; if the regional rationality judgment results in a reasonable regional segmentation result, then outputting the result of the joint review as a qualified display; otherwise, outputting the result of the joint review as an unqualified display.

[0074] The comprehensive audit includes, in the process of judging regional rationality, excluding all rows judged to be reasonable from the display area. If the audit result of the feature consistency in the display area after elimination is that the display is reasonable, then the result of the joint audit is directly output as qualified display; otherwise, the result of the comprehensive audit is generated as unreasonable display.

[0075] The proposed "regional rationality review mechanism" aims to address the issue of overall consistency failure during actual display audits due to the diverse characteristics and complex structures of items. During the standard review process, inconsistencies in the characteristics of a particular group of items can easily lead to entire rows or areas being deemed "irrational," while overlooking well-structured, rational display units within them.

[0076] In this embodiment, the neural network used to analyze the rationality of display area segmentation results is preferably a convolutional neural network (CNN) structure. This neural network, fed with a combined image of the divided sub-regions or their corresponding structured feature matrix, performs a binary classification judgment on whether the segmentation results meet visual and structural rationality. Pre-trained on a dataset containing a large number of display images and their corresponding rationality labels, the CNN model possesses strong feature extraction and classification capabilities, accurately identifying structural damage, feature conflicts, or boundary misalignment within the region combination, thus providing effective judgment basis for subsequent review.

[0077] In an optional embodiment, the above-mentioned neural network can also be any one of the following structures or a combination thereof: multi-layer perceptron (MLP), graph neural network (GNN), attention mechanism network (such as Transformer structure), fusion model (CNN+LSTM or CNN+Transformer) and other model structures.

[0078] S4: If the result of the joint audit is that the display is reasonable, the result of the joint audit is directly output; if the result of the joint audit is that the display is unreasonable, an adjustment strategy is generated.

[0079] The adjustment strategy includes adjusting the display of items so that the result of the joint audit is reasonable and outputting a solution with the shortest adjustment path. The strategy is generated using an optimization algorithm based on energy conservation:

[0080] The energy conservation-based optimization algorithm includes assuming that each displayed item is an indivisible energy and that the energy is rearranged within the display area to achieve a global steady state. The display of items that has passed the review and is judged to be reasonably arranged indicates that the global steady state is achieved.

[0081] In the display area, display items are removed at any position. After removing D display items, if the global state is stable, the positions and item marks of D display items are recorded. By testing different removal strategies, the positions and marks of the display items corresponding to the minimum value of D are selected.

[0082] Recorded as: minimum number of eliminations , the corresponding location set of displayed items , corresponding to the mark of the displayed items .

[0083] in, Indicates the display position of the rth elimination, Indicates the mark of the rth item to be removed; Indicates the number of items that were rejected.

[0084] The specific steps of the rearrangement are:

[0085] Step 1: Assign the remaining display positions to W and get , as an arrangable position.

[0086] Step 2: Make the elements in Q Randomly arrange the objects (each arrangement method is a layout scheme), and analyze whether the global structure is in a steady state after arrangement.

[0087] Step 3: If a steady-state arrangement exists, calculate the sum of the distances L before and after adjustment of each item, and output the arrangement with the smallest L. If a steady-state arrangement does not exist, randomly select one of the displayed items that have not been eliminated as the eliminated display item, repeat the analysis of steps 1 and 2, and after going through all the eliminated display items, output the arrangement with the smallest L.

[0088] If no output is obtained by taking any of the displayed items that have not been eliminated, the number of items taken will be increased (after each increase in the number, all solutions will be explored; if no solution can be output, the number will be increased again) until a solution is output.

[0089] The "energy conservation-based optimization algorithm" proposed in this proposal aims to address the problem of discovering unreasonable structures during display audits but failing to automatically adjust them using simple rules. Traditional adjustment strategies often rely on fixed templates or manual experience, making them difficult to adapt to the complex adjustments required by display areas due to mixed categories, uneven product quantities, and tight layouts. In particular, when the goal is to minimize intervention and restore the optimal layout, a mathematical and systematic strategy generation mechanism is lacking.

[0090] To this end, this invention incorporates the concept of "energy conservation" into its optimization modeling. Each displayed item is considered an indivisible "energy unit." The currently approved reasonable state is defined as the "system steady state," and the display area is abstracted as a system with balanced energy distribution. By removing some energy (i.e., a small number of items) and rearranging the remaining energy in the empty spaces, a new steady-state structure is found, achieving an optimized transition from an unreasonable state to a reasonable one.

[0091] The core goal of this optimization algorithm is to find the minimum set of culls, D, to restore structural rationality and ensure that the adjustment plan minimizes disruption to the original display. A combinatorial search constructs candidate arrangements and explores the optimal solution by traversing culling and rearrangement paths. Using the sum of item movement distances, L, as an evaluation metric, the algorithm selects the path with the minimum overall displacement, balancing rationality and adjustment cost. An iterative incremental strategy is introduced to automatically expand the culling range when initial culling fails, ensuring the final solution is outputtable. The algorithm converges on the minimum adjustment path, making the adjustment strategy interpretable, stable, and operational.

[0092] Through this strategy design, the system can automatically generate adjustment plans that meet the global structural stability without relying on hard rule templates, and can adapt to multi-dimensional characteristics such as product type, level, and location, greatly improving the flexibility, intelligence and engineering practicality of the adjustment logic.

[0093] On the other hand, this embodiment also provides an artificial intelligence-based display item review system, which includes:

[0094] The acquisition unit obtains images of items displayed in the display area and identifies the display position of each displayed item based on the image content.

[0095] The audit unit audits the items at each display location using the first standard of the display rules; if the audit result is that the display is reasonable, the audit result is directly output; if the audit result is not that the display is reasonable, the displayed items are audited according to the second standard based on the audit result of the first standard.

[0096] The analysis unit generates a joint audit result based on the results of the two audits; if the result of the joint audit is that the display is reasonable, the analysis unit directly outputs the joint audit result; if the result of the joint audit is that the display is unreasonable, the analysis unit generates an adjustment strategy.

[0097] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0098] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0099] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, and then editing, interpreting, or processing in another suitable manner as necessary, and then storing it in a computer memory.

[0100] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for reviewing displayed items based on artificial intelligence, characterized in that: include: Obtain an image of items displayed in a display area and identify the display location of each item based on the image content; Using the first criterion of the display rules, the items on each display position are reviewed; if the review result is that the display is reasonable, the review result is directly output; If the audit result is not that the display is reasonable, the displayed items will be audited according to the second standard based on the audit result of the first standard; Generate a joint audit result based on the results of the two audits; if the result of the joint audit is that the display is reasonable, directly output the joint audit result; if the result of the joint audit is that the display is unreasonable, generate an adjustment strategy; The adjustment strategy includes adjusting the display of items so that the result of the joint audit is that the display is reasonable, and outputting a plan with the shortest adjustment path; The adjustment strategy also includes generating a strategy using an optimization algorithm based on energy conservation; The optimization algorithm based on energy conservation includes assuming that each displayed item is an indivisible energy; In the display area, the whole system is kept in a steady state by rearranging the energy; If the display of items is judged to be reasonable after review, it means that the overall situation is in a stable state; In the display area, items are removed at any position. After removing D items, if the global state is stable, the positions and item marks of D items are recorded. By testing different removal strategies, the position and mark of the corresponding items are selected when D is the minimum value. Recorded as: minimum number of eliminations , the corresponding location set of displayed items , corresponding to the mark of the displayed items ; in, Indicates the display position of the rth elimination, Indicates the mark of the rth item to be removed; Indicates the number of items removed; The specific steps of the rearrangement are: Step 1: Assign the remaining display positions to W and get , as an arrangable position; Step 2: Make the elements in Q Randomly arrange the data, and then analyze whether the global data is in a steady state; Step 3: If there is a steady-state arrangement, calculate the sum of the distances L before and after adjustment of each item, and output the arrangement with the smallest L. If there is no steady-state arrangement, randomly select one of the displayed items that have not been eliminated as the eliminated display item, repeat the analysis of steps 1 and 2, and after going through all the eliminated display items, output the arrangement with the smallest L. If no output can be obtained by randomly selecting one of the displayed items that have not been eliminated, increase the number of randomly selected items until the output is obtained.

2. The artificial intelligence-based display item auditing method according to claim 1, characterized in that: The image content recognition results include identification of display item locations, display item defects, and display item types through a trained object detection model; The recognition process of the target detection model is divided into two stages: in the first stage, the position and type of displayed items are directly identified using the target detection model; In the second stage, the types of items displayed at each position are used to match the corresponding attention weights through a preset mapping relationship; Allocating attention to the target detection model according to the attention weight; Using the object detection model after attention allocation, defects are identified for the displayed items at each position.

3. The artificial intelligence-based display item auditing method according to claim 2, characterized in that: After the identification is completed, the defects of each displayed item are recorded, and the defect identification results are output at the same time as the audit results.

4. The artificial intelligence-based display item auditing method according to claim 3, characterized in that: The first criterion includes the review of the display according to the principle of consistency of pre-selected characteristics; Based on the features pre-selected during human-computer interaction before the review, if the pre-selected features are non-numerical features, the non-numerical pre-selected features will be quantified and the review will be conducted based on the quantified pre-selected features; For each row in the display area, feature consistency is reviewed: for each row, the maximum and minimum feature values ​​are screened to ensure that the minimum feature value of the previous row is greater than or equal to the maximum feature value of the current row in one direction or from the middle to both sides; and within each row, the feature values ​​of the items decrease in one direction or from the middle to both sides. When the audit of the consistency of the characteristics is met, the audit result is judged to be reasonable display.

5. The artificial intelligence-based display item auditing method according to claim 4, characterized in that: The second standard includes inheriting the pre-selected features in the first standard and conducting a reasonableness review of the display; wherein the reasonableness is categorized into: regional reasonableness and supplementary reasonableness; The supplementary rationality is achieved through quantitative analysis. When the pre-selected features are non-numerical features, in the nth row, if the number of remaining display positions after the items with feature i are displayed is equal to the number of items with feature j, then the nth row is judged to meet the rationality; otherwise, the nth row is judged to not meet the rationality. Among them, i and j represent different feature indexes in the same feature; The regional rationality is achieved by segmenting the display area: within the display area, the display area is divided into sub-areas to obtain sub-area combinations, so that the display of items in each sub-area satisfies the feature consistency review, or satisfies the combined review of the feature consistency and the supplementary rationality; the sub-area combination with the least sub-areas is output as the segmentation result of the display area; Through the pre-trained neural network, the rationality of the segmentation results of the display area is analyzed, and a binary classification judgment is obtained to determine whether the regional segmentation results are reasonable.

6. The method for display item review based on artificial intelligence according to claim 5, characterized in that: The joint review includes, firstly based on the regional rationality, conducting a comprehensive review of the feature consistency and the supplementary rationality; if the result of the comprehensive review is that the display is unreasonable, then judging the regional rationality; If the judgment result of the regional rationality is that the regional segmentation result is reasonable, the result of the joint review is output as qualified display; otherwise, the result of the joint review is output as unqualified display; The comprehensive audit includes, in the process of judging regional rationality, eliminating all rows judged to be reasonable in the display area. If the audit result of the feature consistency in the display area after elimination is that the display is reasonable, then the result of the joint audit is directly output as qualified display; otherwise, the result of the comprehensive audit is generated as unreasonable display.

7. An artificial intelligence-based display item review system using the method according to any one of claims 1 to 6, characterized in that: The acquisition unit acquires images of items displayed in the display area and identifies the display position of each item based on the image content; The review unit reviews the items at each display location using the first standard of the display rules; if the review result indicates that the display is reasonable, the review result is directly output; If the audit result is not that the display is reasonable, the displayed items will be audited according to the second standard based on the audit result of the first standard; The analysis unit generates a joint audit result based on the results of the two audits; if the result of the joint audit is that the display is reasonable, the analysis unit directly outputs the joint audit result; if the result of the joint audit is that the display is unreasonable, the analysis unit generates an adjustment strategy.

8. A computer device comprising: memory and processor; The memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images