Configuration method of image processing unit and its electronic device and storage medium

Abstract

This invention relates to the field of image recognition result processing. One embodiment of the invention provides a method for configuring computational units for product images, comprising providing multiple computational units through a user interface; and, based on user operations, associating the multiple computational units to establish a processing flow for the multiple computational units, wherein the processing flow is directional and does not form a loop / closed loop. This improves the processing capability for non-standardized computations of image recognition results for standardized products.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a configuration method for image processing nodes. Background Technology

[0002] A Stock Keeping Unit (SKU) is a unique identifier for each product and service. The system uses SKU values ​​for data management, enabling companies to track inventory levels in systems such as warehouses and retail stores or for products.

[0003] SKUs serve as identifiers for products on shelves within a shelf structure. When each SKU is linked to an item, variant, product line, bundle, service, charge, or seizure, the individual item doesn't always refer to the actual physical item; more accurately, it refers to the billing entity. Different businesses use SKUs differently, and their SKU allocation systems depend on the company's data storage and retrieval strategies in their region or country. SKU tracking systems may vary due to broader control methods such as regulations and rules of manufacturers or third-party agencies. A successful inventory management system assigns a unique SKU to each product and its variants (such as different versions, styles, or bundles), including the quantity and model information of the related products. This allows businesses to effectively track sales, such as determining whether a blue shirt or a green shirt sells better. Other entities may also follow different guidelines regarding tracking methods.

[0004] Image recognition refers to the technology of using computers to process, analyze, and understand images in order to identify targets and objects of various patterns. It is a practical application of deep learning algorithms. Currently, image recognition technology is generally divided into facial recognition and product recognition. Facial recognition is mainly used in security checks, identity verification, and mobile payments; product recognition is mainly used in the process of commodity circulation, especially in unmanned retail fields such as unmanned shelves and smart retail cabinets.

[0005] The traditional image recognition process consists of four steps: image acquisition → image preprocessing → feature extraction → image recognition. Internationally, Cognex is a leading provider of image recognition software, while domestically, companies like Tuiyi and Haishen Technology are prominent examples. Additionally, in geography, the term refers to the technique of classifying remote sensing images.

[0006] Image recognition is the process by which visual stimuli act on sensory organs, and people recognize them as a familiar image. It requires both information entering the senses at the time of perception and information stored in memory. Only through the processing of comparing stored information with current information can image recognition be achieved.

[0007] Humans possess a remarkable ability to recognize images. Changes in image distance or the position of the image on the sensory organs can alter the size and shape of the image on the retina. Even under these circumstances, people can still recognize images they have perceived in the past. Furthermore, image recognition can transcend sensory limitations. For instance, a person can see words and recognize them when someone writes on their back.

[0008] Image recognition is likely based on the main features of an image. Every image has its features; for example, the letter A has a point, the letter P has a circle, and the center of the letter Y has an acute angle. Eye movement studies during image recognition show that the gaze is always focused on the main features of the image, that is, on the areas where the curvature of the image contour is greatest or where the contour direction changes abruptly—these areas contain the most information. Furthermore, the eye's scanning path always moves sequentially from one feature to another. Therefore, in the process of image recognition, the perceptual mechanism must filter out redundant input information and extract key information. Simultaneously, there must be a mechanism in the brain responsible for integrating information, organizing the information acquired in stages into a complete perceptual image.

[0009] In human image recognition systems, the recognition of complex images often requires processing information at different levels. For familiar graphics, because their main features are understood, they are recognized as a single unit, and details are no longer considered. This holistic unit composed of isolated unit materials is called a chunk, and each chunk is perceived simultaneously. In the recognition of written materials, people can not only group the strokes or radicals of a Chinese character into chunks, but also group frequently occurring characters or words into chunk units for recognition.

[0010] In computer vision recognition systems, image content is typically described using image features. In fact, computer vision-based image retrieval can also be divided into three steps similar to those of a text search engine: feature extraction, index building, and querying.

[0011] Image recognition is an important field of artificial intelligence. To develop computer programs that simulate human image recognition activities, different image recognition models have been proposed. One such model is the template matching model. This model posits that recognizing an image requires a memory of that image pattern in past experience, also known as a template. If the current stimulus matches the template in the brain, the image is recognized. For example, given the letter A, if there is a template A in the brain whose size, orientation, and shape perfectly match the template, the letter A is recognized. This model is simple, straightforward, and easily applied in practice. However, this model emphasizes that the image must perfectly match the template in the brain for recognition. In reality, humans can recognize not only images that perfectly match the template but also images that do not perfectly match it. For example, people can recognize not only a specific letter A but also printed, handwritten, misaligned, and differently sized versions of the letter A. Furthermore, the number of images humans can recognize is vast; it would be impossible for every image to have a corresponding template in the brain.

[0012] To address the problems of template matching models, Gestalt psychologists proposed a prototype matching model. This model posits that long-term memory stores not countless templates for recognition, but rather certain "similarity" characteristics of images. These abstracted "similarity" characteristics serve as prototypes, used to test the images to be recognized. If a similar prototype is found, the image is recognized. This model is more suitable than template matching models from a neural and memory retrieval perspective, and it can also explain the recognition of irregular images that are similar to prototypes in certain aspects. However, this model does not explain how humans distinguish and process similar stimuli, and it is difficult to implement in computer programs. Therefore, a more complex model, the "pan-magic" recognition model, has been proposed.

[0013] In general industrial applications, industrial cameras are used to capture images, and then software is used to process the images based on grayscale differences to identify useful information. Representative image recognition software companies abroad include Cognex, while domestic companies include Tui Intelligent.

[0014] The development of image recognition has gone through three stages: character recognition, digital image processing and recognition, and object recognition. Research on character recognition began in 1950, generally involving the recognition of letters, numbers, and symbols. From printed character recognition to handwritten character recognition, its applications are very wide-ranging.

[0015] Research on digital image processing and recognition began in 1965. Compared to analog images, digital images offer significant advantages, including convenient storage and transmission, compressibility, reduced distortion during transmission, and ease of processing. These advantages have provided a powerful impetus for the development of image recognition technology. Object recognition primarily refers to the perception and understanding of objects and their environment in the three-dimensional world, falling under the advanced category of computer vision. It is a research direction based on digital image processing and recognition, combining artificial intelligence, systems science, and other disciplines. Its research results are widely applied in various industrial and exploration robots. One shortcoming of modern image recognition technology is its poor adaptive performance; once the target image is heavily contaminated with noise or has significant defects, it often fails to yield ideal results.

[0016] The mathematical essence of image recognition is a mapping problem from pattern space to category space. Currently, there are three main recognition methods in the development of image recognition: statistical pattern recognition, structural pattern recognition, and fuzzy pattern recognition. Image segmentation is a key technology in image processing. Its research has a history of several decades, dating back to the 1970s, and it has always received high attention. To date, thousands of segmentation algorithms have been proposed using various theories, and research in this area continues actively.

[0017] There are many existing image segmentation methods, including thresholding, edge detection, region extraction, and methods combining specific theoretical tools. Based on image type, they can be categorized as grayscale image segmentation, color image segmentation, and texture image segmentation. Edge detection operators were proposed as early as 1965, leading to numerous classic edge detection algorithms. However, in the last two decades, with the rapid development of computational techniques for image segmentation based on histograms and wavelet transforms, and VLSI technology, significant progress has been made in image processing research. Image segmentation methods combine specific theories, methods, and tools, such as mathematical morphology-based image segmentation, wavelet transform-based segmentation, and genetic algorithm-based segmentation.

[0018] Regarding the application of image processing in retail and other commercial sectors, patent consultant Zhao Fenglin's research revealed several existing technologies. Most of these involve manual processing and labeling of images captured from store shelves and smart shelves to help identify out-of-stock items. Furthermore, the accompanying electronic processing workflows are relatively rigid and inefficient. For example, patent application CN111061890A mentions that due to the high loss rate of unmanned shelves and the high cost of vending machines, smart vending machines are gradually becoming a new trend in the unmanned retail field. Smart vending machines generally adopt cashless transactions and ordinary currency transactions. Examples include self-service orange vending machines, self-service coffee machines, self-service ice cream machines, and unmanned vending machines. Due to the unmanned nature of smart vending machines, restocking personnel are required to regularly check and replenish stock to ensure that the smart vending machines can meet users' purchasing needs at any time.

[0019] Currently, smart vending machines are equipped with cameras that periodically take pictures of the stock keeping units (SKUs) of the products inside the vending machine. These pictures are then uploaded to a server, where staff label the SKUs based on the images to obtain the product labeling information. The SKU retrieval database is then updated based on this labeling information.

[0020] However, the large number of SKU images and their high similarity make labeling errors easy, resulting in low accuracy of the data stored in the SKU retrieval database.

[0021] Patent application CN111061890A also mentions that the embodiments of this application can be applied to scenarios where it is necessary to verify the annotation information in images. The aforementioned images can be top-view images of the product, front-view images of the product, and partial images of the product. Specifically, as an example, in a shelf compliance inspection scenario, in order for brand owners to analyze product display area ratio, key SKU distribution rate, out-of-stock rate, and pass rate, it is necessary to identify the quantity and type of different products displayed on shelves, freezers, or endcaps. Therefore, during the identification process, it is necessary to verify the annotation information corresponding to different products in the images acquired by the imaging device, so as to accurately identify the quantity and type of different products. As another example, in a self-checkout scenario, when performing self-checkout, an imaging device can be used to acquire the product to be checked out, and then the product can be automatically identified. Based on the identification result, the payment system can then settle the payment for the product. Therefore, during the identification process, it is necessary to verify the annotation information corresponding to the product in the images acquired by the imaging device, so as to accurately identify the SKU corresponding to the product, thereby combining with the imaging device and the payment system to complete accurate self-checkout. As another example, in interactive marketing scenarios, where only product images are submitted via the user interface, it's necessary to identify the SKU corresponding to the submitted product image to complete different interactive marketing activities in accordance with game rules and improve the accuracy of product search. As yet another example, in smart vending machines, a camera is needed to capture the purchased product, then automatically identify it, and finally settle the payment based on the identification result. Therefore, image annotation and training are required for the products sold in the vending machine, enabling verification of the labeled information when purchasing goods, thus achieving accurate automatic identification and settlement of purchased goods in the smart vending machine. It should be understood that smart vending machines can come in various types, such as coin-operated vending machines located in subway stations or large shopping malls, or unmanned smart vending machines located in offices and unmanned convenience stores. That is, the method of verifying labeled information in unmanned smart vending machines can be applied not only to drop-down vending machines but also to door-opening unmanned smart vending machines.

[0022] In existing technologies, most solutions focus on addressing stockouts on shelves and product stocking / replenishment.

[0023] Furthermore, the process of reviewing the display of goods on store shelves can only be done manually. The review rules are written into SQL or other program code using hard coding, and the image recognition results are then calculated, queried, and filtered to obtain the desired data. The data collection process is slow and consumes a lot of manpower. Summary of the Invention

[0024] Based on the inventor's research into potential needs: if the configurability of the software processing flow for image recognition results of goods / products can be improved, it will be beneficial for scenarios such as product display verification / inspection.

[0025] According to one aspect of the present invention, the configuration method of the usual hard-coded program is transformed into two aspects: 1) a more flexible modular configuration for the processing flow of data reflecting product display, and 2) the sequential execution of the functions of multiple computing units and the output of one or more results of the processing flow according to the configuration order of the modular functional units in the processing flow.

[0026] Based on this concept, an embodiment of the present invention proposes a method for configuring a computing unit for product display, comprising two steps:

[0027] S-1) provides multiple computing units through a user interface, which are independent program modules;

[0028] S-2) Based on the user's interactive operations, associate multiple computing units to establish a processing flow for multiple computing units.

[0029] Optionally, multiple computing units are cascaded hierarchically. User operations can be performed through a (visual) user interface or via non-visual methods such as command line. This allows users to dynamically configure the individual attributes of multiple computing units or their interconnections through the user interface, rather than hard-coding the processing flows related to these computing units. This improves the customizability and scalability of the processing flows related to these computing units, or in other words, their adaptability to different application scenarios and their efficiency.

[0030] Optionally, in some embodiments, multiple computing units each have interfaces for input and / or output, and these interfaces are all displayed to the user through a user interface for connection or configuration operations. Step S-2) further includes: in response to the user's connection operation, connecting multiple computing units sequentially through the interfaces; wherein, along the direction of the processing flow, at least some of the output interfaces of the multiple computing units are connected to the input interfaces of adjacent computing units.

[0031] Optionally, in some embodiments, the plurality of computing units includes a first computing unit having a first output interface, and the configuration method further includes: in response to a user's connection operation, connecting the first output interface to the input interface of a second computing unit among the plurality of computing units.

[0032] Optionally, in some embodiments, where the first processing unit only has a first output interface, the method further includes the step of: setting the attributes / parameters of the first processing unit according to the user's configuration operation, so as to apply them to the second processing unit or processing flow through the first output interface. Here, the first processing unit provides a configuration interface for the user, allowing the user to configure and change the operation / running of processing flows related to multiple processing units by configuring the attributes / parameters of the first processing unit, thereby improving the customizability of the processing flow and its adaptability to application scenarios.

[0033] Optionally, in some embodiments, the first computing unit can serve as the overall data input interface for the processing flow formed by the computing units, and can be used to import data reflecting the structured / hierarchical / layered product display for the processing flow.

[0034] Optionally, in some embodiments, the data includes image recognition data of an image element array or questionnaire data; and multiple processing units, each representing a basic operation, or minimum processing unit, or operator, for the image element array. Alternatively, multiple processing units may each represent a basic operation for the questionnaire data. Or, multiple processing units may each represent a basic operation capable of processing both the questionnaire data and the image element array.

[0035] The first processing unit can also execute the attributes of image recognition data or questionnaire data imported into the processing flow. If the image recognition data is for freezer displays, the attributes of the image recognition data may include: the type / geographical location of the store where the freezer is located, whether the freezer is located in a supermarket, etc.

[0036] Optionally, in some embodiments, the image element array includes rows / columns of image elements corresponding to the rows of products displayed on shelves, containers, freezers, or floor stacks.

[0037] It is understandable that image recognition data is generated by recognizing raw image data taken from shelves, counters, freezers, or floor stacks. The array of image elements in this image recognition data also corresponds to the goods / products on the shelves, counters, freezers, or floor stacks. For example, each image element in the array has location information, corresponding to a product on the shelf, counter, freezer, or floor stack. After preprocessing such as cleaning the raw image data, the SKU information of the corresponding products can also be associated with each image element; correspondingly, the image recognition data after this preprocessing can also be called SKU image (data).

[0038] Optionally, in some embodiments, the basic operations include at least one or more of the following: logical operations, mathematical operations, array operations, topological relationship operations, positional relationship operations, and corresponding product SKU-related operations on the image element array or one or more image elements therein. The topological relationship operations, positional relationship operations, and at least some of the SKU-related operations are based on the position of each image element in the image element array.

[0039] Optionally, in some embodiments, at least one of i) SKU-related operations, ii) logical operation units, mathematical operation units, and array operation units is located at an upper / upstream level in the processing flow relative to iii) topological relationship operation units and / or iiii) positional relationship operation units. In other words, the topological relationship operation units and positional relationship operation units need to process SKU information based on image elements and process image elements in the image element array in an array manner or logical manner.

[0040] Optionally, in some embodiments, the processing flow forms a directed acyclic graph. This improves readability during configuration and enhances the processing efficiency, for example, by preventing infinite loops.

[0041] Optionally, in some embodiments, SKU-related operations include at least one of the following three:

[0042] I) Filter the SKU list: Define filter criteria and / or target SKUs.

[0043] II) Grouping by conditions: Define grouping conditions and group according to those conditions.

[0044] III) Calculate the number of facets: Calculate the number of facets for image elements with the target SKU.

[0045] Optionally, in some embodiments, the positional relationship calculation includes: filtering the centralized display of target SKUs, and / or obtaining a combination of multiple adjacent centralized displays of target SKUs; wherein obtaining a combination of multiple adjacent centralized displays of target SKUs includes: determining whether the centralized display for the first SKU and the centralized display for the second SKU are adjacent.

[0046] Optionally, in some embodiments, the logical operation includes at least one of logical AND, logical OR, and logical NOT operations; and

[0047] Mathematical operations include at least one of addition, subtraction, multiplication, and division; and

[0048] Array operations include at least one of the following:

[0049] Find the length of the array.

[0050] Check if the array length is greater than 0.

[0051] Merge arrays: merge arrays

[0052] Array In operation: Performing the In operation on an array

[0053] Summing the values ​​in an array: Sum the values ​​in an array.

[0054] Basic array sorting: Sorts the array in ascending or descending order.

[0055] Array is all true: Check if the array is all true

[0056] Determine if any element in the array is true:

[0057] Get the maximum value in the array

[0058] Find the N maximum values ​​in an array. Find the Top N values ​​in an array. Find any N values ​​in the array.

[0059] List traversal: Replacement: Replace one characteristic of an object with another characteristic.

[0060] List filtering and traversal: filtering based on specific conditions

[0061] List sorting and traversal: Sort the list according to specified criteria.

[0062] Optionally, in some embodiments, the user interface includes a menu area and a configuration area;

[0063] Step S-1) also includes:

[0064] Multiple processing units are categorized and provided in the menu area;

[0065] Based on the user's drag / selection operation, the corresponding calculation unit in the menu area is provided in the configuration area of ​​the interface;

[0066] Step S-2) further includes: in response to the user's connection operation, establishing a connection between the output interface and the input interface of the corresponding arithmetic unit to allow the corresponding parameter transmission.

[0067] Optionally, some embodiments of the method further include the step of: in response to a user's selection and save operation of at least two computational units associated in the processing flow, template the at least two computational units and save them to the menu area.

[0068] For example, in response to a user's combination operation on multiple operation units in the configuration canvas area, the selected operation units are combined and configured, and the combination is saved in the menu area so that the user can directly access and use this combination of operation units later, without having to edit and generate the combination every time it is needed. Therefore, in subsequent user operations, the number of times operation units are selected or moved is reduced, and the structure of the directed acyclic graph of the processing logic is more concise.

[0069] Optionally, in some embodiments, the SKU-related operations further include at least one of the following:

[0070] Get POSM list: Get the material list for the corresponding product;

[0071] Get Attribute: Retrieves a specific attribute of the corresponding product;

[0072] Get Price: Get the price of the corresponding product;

[0073] Is there price communication? Determine if the corresponding product has a price tag;

[0074] In addition, basic operations also include:

[0075] Dynamic scene filtering: Determine / filter a set of scenes;

[0076] Get Scene Type: Get the type of the scene.

[0077] Optionally, some embodiments of the configuration method further include step S-3): importing image recognition data and / or questionnaire data into the processing flow; and,

[0078] Step S-4): Run the processing flow to process image recognition data or questionnaire data.

[0079] Through steps S-3) and S-4), based on the configuration of each node in the data processing flowchart and the connection with adjacent nodes, the corresponding functions of each unit are activated along the processing flow, data / parameters are transmitted, and the final processing results for image recognition data and / or questionnaire data are presented to the user.

[0080] Optionally, in some embodiments of the configuration method, preprocessing such as recognition is performed on the raw image data from the terminal device to obtain image recognition data. Step S-4) further includes:

[0081] In response to the operation of running the processing flow, the functions corresponding to multiple computing units in the processing flow are run sequentially to process image recognition data; and the method also includes step S-5): feeding back the processing result of the image recognition data to the terminal device.

[0082] Of course, the multiple computing units in some embodiments can be understood as a breakdown of the product display definition / rules preset by the brand owner. Therefore, optionally, in some embodiments, if the brand owner's definition of its target product display is obtained through a formatted data collection form, step S-0) may be included before step S-1), where the formatted product display definition is broken down / parsed at the server or terminal device to obtain at least a portion of the multiple computing units.

[0083] In another embodiment of the present invention, a method for configuring a processing unit for an image is proposed, comprising: step S-1) and step S-2).

[0084] S-1) provides multiple processing units through a user interface;

[0085] S-2) Based on user operations, multiple computational units are associated to establish a processing flow (operation logic) for these units. This processing flow is directional and does not form loops / closed loops. Alternatively, based on user operations, multiple computational units are associated to establish a directed acyclic graph (DAG) of the image processing flow.

[0086] Optionally, multiple processing units can be cascaded hierarchically. User operations can be performed through a visual user interface or through non-visual methods such as command line input.

[0087] Optionally, in some embodiments, the plurality of computing units each have an interface for input and / or output; and step S-2) further includes: in response to a user's connection operation, sequentially associating the plurality of computing units through the interface; wherein, along the direction of the processing flow, at least a portion of the output interfaces of the plurality of computing units are connected to the input interfaces of adjacent computing units.

[0088] Optionally, connections are formed between the corresponding nodes of each computational unit in the configuration area for multiple computational units to generate a directed acyclic graph.

[0089] Optionally, in some embodiments, the plurality of computing units includes a first computing unit having only a first output interface, and the configuration method further includes: in response to a user's connection operation, connecting the first output interface to the input interface of a second computing unit among the plurality of computing units.

[0090] Optionally, in some embodiments, the method further includes the step of: setting the attributes / parameters of the first computing unit according to the user's configuration operation, so as to apply them to the second computing unit or processing flow through the first output interface.

[0091] For example, in some embodiments, a configuration step for the computing unit is also included: in response to selecting a computing unit in the configuration area (e.g., by right-clicking the mouse), a configuration interface for that computing unit is displayed; and based on the user's configuration operation on the configuration interface, the relevant attributes / functions / parameters of the corresponding computing unit are configured. This configuration step applies not only to intermediate computing units in directed acyclic processing flows / logic, but also to the initial computing units in processing flows / logic. This allows the user to intervene manually at any stage of the entire processing flow to change the operation mode of the entire processing flow / logic, thereby further improving the flexibility of the entire processing flow and the responsiveness to changes in the processing flow to adapt to product display inspection rules.

[0092] Optionally, in some embodiments, step S-3) is further included: importing the recognition data for the image into the processing flow; wherein the image includes hierarchical elements, and,

[0093] Step S-4): Run the processing flow to process the image recognition data.

[0094] Optionally, in some embodiments, the hierarchical elements are products arranged within the accommodating structure; and multiple computing units, each having defined input and output parameters for certain computing rules, which optionally correspond to a basic / minimum computing operation for a product SKU.

[0095] Optionally, in some embodiments, the accommodating structure is a shelf, display case, or freezer; and the image includes image elements corresponding to products arranged in rows / columns within the shelf, display case, or freezer. Alternatively, the image with hierarchical image element arrangement is an image of a shelf or freezer displaying goods / items. These goods / items have SKUs, so the image elements of these goods / items in the image also correspond to SKU information, and each image element of these goods / items also has positional information within the image.

[0096] Optionally, in some embodiments, the basic / minimum operations include at least one or more of the following: logical operations, mathematical operations, array operations, topological relationship operations, positional relationship operations, and SKU-related operations for the product SKU.

[0097] Topological relationship operations, positional relationship operations, and at least some SKU-related operations, based on the positional information of the product's image elements in the image.

[0098] Optionally, in some embodiments, at least one of i) SKU-related operations, ii) logic operation unit, mathematical operation unit, and array operation unit, and at least one of iii) topological relationship operation unit and positional relationship operation unit are cascaded sequentially in the processing flow.

[0099] Optionally, in some embodiments of the configuration method, the SKU-related calculation includes at least one of the following three:

[0100] I) Filter the SKU list, including defining filter criteria and / or filtering the desired SKUs.

[0101] II) Grouping by conditions: Define grouping conditions and group according to those conditions.

[0102] III) Calculate the number of SKU display spaces: Calculate the number of SKU display spaces;

[0103] The positional relationship calculation includes: filtering the centralized display of SKUs, and / or obtaining a combination of adjacent centralized display sets; wherein obtaining a combination of adjacent centralized display sets includes determining whether the first SKU display and the second SKU display are adjacent.

[0104] Optionally, in some embodiments of the configuration method, the logical operation includes at least one of logical AND, logical OR, and logical NOT operations; and

[0105] Mathematical operations include at least one of addition, subtraction, multiplication, and division.

[0106] Optionally, in some embodiments of the configuration method, array operations include at least one of the following:

[0107] 1) Find the length of the array.

[0108] 2) Check if the array length is greater than 0.

[0109] 3) Merge arrays: Merge arrays

[0110] 4) Array In operation: Performing the In operation on an array

[0111] 5) Summing the values ​​in an array: Summing the values ​​in an array.

[0112] 6) Basic array sorting: Sorts the array in ascending or descending order.

[0113] 7) All true values ​​in the array: Check if all true values ​​in the array are true.

[0114] 8) All values ​​in the array are true: Determine if any value in the array is true.

[0115] 9) Find the maximum value in the array

[0116] 10) Find the N maximum values ​​in an array: Find any value (any value is N) in the array.

[0117] 11) List traversal: Replacement: Replace an object's characteristic with another characteristic.

[0118] 12) List traversal: Filtering: Filtering according to specific conditions

[0119] 13) List traversal: Sorting: Sort according to specified conditions

[0120] Optionally, in some embodiments of the configuration method, the user interface includes a menu area and a configuration area;

[0121] Step S-1) also includes:

[0122] Multiple processing units are categorized and provided in the menu area;

[0123] Based on the user's drag / selection operation, the corresponding calculation unit in the menu area is provided in the configuration area of ​​the interface;

[0124] Step S-2) further includes: in response to the user's connection operation, establishing a connection between the output interface and the input interface of the corresponding arithmetic unit to allow the corresponding parameter transmission.

[0125] Optionally, the configuration method in some embodiments further includes the step of: in response to the user's selection and save operation of at least two computation units associated in the processing flow, template the at least two computation units and add them to the menu area.

[0126] Optionally, in some embodiments, the image recognition data comes from a terminal device, and step S-4) further includes: in response to the operation of running the processing flow, sequentially running the functions corresponding to multiple computing units in the processing flow to process the image recognition data. For example, according to each computing unit in the image recognition result processing flow and the configuration therebetween, running the program corresponding to each computing unit or activating its processing function.

[0127] The method also includes step S-5): feeding back the processing results of the image recognition data to the terminal device.

[0128] Optionally, in some embodiments, the SKU-related operations further include at least one of the following:

[0129] Get POSM list: Get the material list;

[0130] Get property: Specify a single property of the object;

[0131] Get Price: Get the product price;

[0132] Was there any price communication? Determine if there is a price tag.

[0133] Dynamic scene filtering: Filters a set of scenes that meet certain criteria;

[0134] Get Scene Type: Get the type of the scene.

[0135] Optionally, in some embodiments, an information processing apparatus for use in an electronic device having one or more processors is also proposed, comprising: means for performing any of the multiple methods of this application or instructions / steps thereof.

[0136] Optionally, in some embodiments, a computer-readable storage medium is also provided for storing one or more programs, the one or more programs including instructions that, when executed by an electronic device having a display and a touch-sensitive surface, cause the device to perform any of the multiple methods of this application or its instructions / steps.

[0137] Optionally, in some embodiments, an electronic device is also proposed, comprising: a display; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions / steps for performing any of the multiple methods in this application or thereof.

[0138] Optionally, in some embodiments, a graphical user interface on an electronic device is also provided, the electronic device having a display, a touch-sensitive surface, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface including the user interface displayed in any of the multiple methods in this application.

[0139] Optionally, in some embodiments, an electronic device is also proposed, comprising: a display; a touch-sensitive surface; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions / steps for performing any of the multiple methods in this application or thereof.

[0140] By inventing methods according to some embodiments, a human-computer interaction interface is provided in the image recognition data processing flow, allowing users to configure the processing flow, thereby making the processing flow more configurable. This allows users to flexibly configure multiple computing units and their connections, more flexibly forming the association logic between the various computing units to adapt to the diverse requirements of inspecting / processing hierarchically arranged image elements in images such as shelves, and improves the processing capability for images with hierarchically arranged image elements, reducing the cycle from image acquisition to processing and feedback of processing results. Attached Figure Description

[0141] For those skilled in the art, the specification provides a complete and practicable disclosure of embodiments of the invention, including its preferred embodiments, wherein the specification refers to the accompanying drawings, which are:

[0142] Figure 1 A schematic diagram illustrating the transmission of image data between a terminal device and a server according to an embodiment of the present invention is shown.

[0143] Figure 2 A flowchart of a method according to an embodiment of the present invention is shown;

[0144] Figure 3 A schematic diagram of a user interface providing multiple computing units and a configuration area of ​​computing units is shown in a method according to an embodiment of the present invention;

[0145] Figure 4 A schematic diagram of one of a plurality of basic operational units in a method according to an embodiment of the present invention is shown;

[0146] Figure 5 It shows Figure 1 A schematic diagram of an identification result IMR_5 from the image acquisition data DS10_1 in the embodiment;

[0147] Figure 6 A configuration interface for setting parameters of a computing unit according to another embodiment of the present invention is shown;

[0148] Figure 7 In one embodiment of the present invention, the process of verifying whether the purity of KO soda in the freezer is greater than or equal to 95% based on image recognition results is included.

[0149] Figure 8 A multi-output processing flow consisting of a number of arithmetic units is shown in another embodiment of the present invention;

[0150] Figure 9 The process for determining whether Red Bull and Monster Energy beverages are displayed adjacent to each other in an image is shown in another embodiment of the present invention.

[0151] Figure 10 It shows Figure 9 The output of the process for determining whether Red Bull and Monster Energy drinks are displayed in adjacent areas;

[0152] Figure 11a , Figure 11b Two types of questionnaire data applicable to some embodiments of the present invention are shown;

[0153] Figure 12a , Figure 12b , Figure 12c , Figure 12d Four illustrative interactive interfaces are shown in some embodiments of the present invention during the template creation process;

[0154] Figures 13a-13c The present invention illustrates methods for performing secondary / multiple calculations and the interactive interfaces involved therein in some embodiments of the present invention;

[0155] The same or similar reference numerals in this specification and drawings represent the same or similar technical features or elements of the present invention. Detailed Implementation

[0156] Figure 1 A schematic diagram illustrating the transmission of image data between a terminal device DS10 and a server SV109 according to an embodiment of the present invention is shown. Currently, in the scenario of shelf_1 compliance inspection, in order for brand owners to analyze the display area ratio of target product Prod_1, whether the product display conforms to the arrangement rules defined by the brand owner, or further determine the distribution rate, out-of-stock rate, and pass rate of key SKUs, etc., it is necessary to take pictures of the products displayed on the shelf_1, freezer, or endcap to obtain raw image data Img_1, and transmit the relevant image data Img_1 to the server SV109.

[0157] Manual review of the display of product Prod_1 on shelf Shelf_1 via server SV109 may rely on software programs. These programs can obtain the SKU data of product Prod_1 displayed on shelf Shelf_1 based on image recognition results, and then determine whether the display of product Prod_1 conforms to the display rules defined by the brand. However, if these software programs are simply hard-coded, each set of display rules defined by the brand, corresponding to one or more calculations, requires separate development. Moreover, the hard-coding work for these display rules generally can only be done manually, writing SQL or programming code to calculate, query, and filter the image recognition results to obtain the desired data. Therefore, the R&D resources invested are large, time-consuming, and inefficient, making it difficult to meet the needs of operational business.

[0158] After researching relevant technologies and identifying these potential needs, the inventors realized that improving the configurability of the processing flow for image recognition results of standardized goods / products Prod_1 would be beneficial for product display verification scenarios.

[0159] The inventors also discovered that, because business is diverse and ever-changing, the corresponding software-based processing flow and methods also need to be flexible. If the commonly used software functional units in the calculation of product display rules can be modularized, preferably into the smallest visual unit, and can be configured on-site by the user through the user interface, then users can quickly and flexibly configure these smallest calculation units for product display rules, and construct the calculation flow and processing logic for product display rules.

[0160] Furthermore, if the basic computing units are provided through the user interface, multiple computing units can be associated simply by configuring connections to correspond to the brand's product display definition rules, eliminating the need to spend a significant amount of time writing, testing, and deploying hard code. This improves the efficiency of the software program in processing image data based on product display definitions.

[0161] The more complete the basic computing units required by the technical concept in this aspect of the invention, the more diverse the program functions can be supported, thereby adapting to the diverse definition rules of product display by brand owners. When the basic computing units are sufficiently complete, users can create different and diverse program flows (e.g., in the form of directed acyclic graphs) on-site to obtain calculation results for more rule definitions of product display for brand owners. Therefore, the efficiency of reviewing store display performance based on image recognition technology can be improved.

[0162] Based on this technical concept, in one embodiment of the present invention, a flexible architecture is designed. This architecture supports configuration through an interface or by writing simple code scripts to obtain processed results from the raw results of image recognition after one or more calculations / processing. It is applicable to the judgment of merchandise / product display, including but not limited to: the existence of SKUs or other items (such as price tags, materials, store signs, etc.), the number of SKUs on the shelf, the price of SKUs, the brand owner of SKUs, the brand of SKUs, the series of SKUs, the category of SKUs, the specifications of SKUs, the flavor of SKUs, the packaging of SKUs, the proportion of specified SKUs, the number of SKUs or flavors, or the category count obtained after classifying according to any attribute, the proportion of specified SKUs on the shelf, the purity / saturation of the freezer, whether SKUs are placed close together or in a concentrated manner, the shelf level or freezer door where the SKU is located, the price of the SKU, the scene (such as freezer, shelf, etc.) or area to which the SKU belongs, and any result obtained after performing AND, OR, NOT or addition, subtraction, multiplication and division operations based on the above results.

[0163] Figure 2 A flowchart of a method according to an embodiment of the present invention is shown.

[0164] In one embodiment of the present invention, a method for configuring a computing unit for product display is proposed, comprising two steps S-1) and S-2):

[0165] In step S-1), multiple computing units are provided as independent, configurable program modules through a user interface presented on the display of server SV109.

[0166] In step S-2), based on the user's interactive operation, multiple computing units are associated to establish a processing flow for the multiple computing units.

[0167] Figure 3 A schematic diagram of a user interface provided with multiple computing units and a configuration area for the computing units is shown in a method according to an embodiment of the present invention. Figure 4 It shows Figure 3 A schematic diagram of one of the multiple operation units Operat_3 in the method of the Chinese embodiment.

[0168] Optionally, based on user interactions such as clicking and dragging multiple operation units (operat_3) in the basic operation unit area (operarea_3) on the left side of the user interface (UI_3), a portion of the multiple operation units (operat_3) are presented and hierarchically / layer-wise cascaded in the configuration area (config_3) on the right side of the user interface (UI_3). In addition to the above-mentioned visual methods, user operations can also be applied to multiple operation units (operat_3) or to a template composed of multiple operation units via command lines, scripts, etc., thereby forming a processing flow (process_3) among the multiple operation units (operat_3) shown in the configuration area (config_3). Alternatively, in response to user commands, scripts, interactive operations, etc., multiple operation units (operat_3) or to a template composed of multiple operation units can be applied, thereby forming further computational associations for the processing flow (process_3), which are presented through other interfaces to accept further user operations. Related embodiments can be found in the embodiments under the title "Secondary / Multiple Operations". This allows users to dynamically configure the individual attributes of multiple operation units (operat_3) or their connections through the user interface (UI_3), instead of configuring the processing flow related to these operation units in a hard-coded manner. This improves the customizability / scalability of the processing flow related to these operation units, or in other words, its adaptability to different application scenarios and its work efficiency.

[0169] From one perspective, some embodiments of the present invention provide a software-based rule engine that allows users to flexibly configure the processing flow (process_3) of the rule engine through human-computer interaction. This allows the recognition results of the original image acquisition data DS10_1 (the recognition process for image acquisition data can be performed via cloud services, servers, or mobile terminals) to be processed through any number of different calculations in the configured processing flow (process_3) to obtain the corresponding results. The relevant embodiments of this invention focus on the architecture of the rule engine and its configuration method, rather than on individual computational rules that can be arbitrarily added to the engine. The specific implementation of the computational rule unit and the functionality of the computational unit (operat_3) does not constitute a limitation of the present invention.

[0170] The operation unit `operat_3` in the processing flow `process_3` can be understood as an operator, the smallest unit of operation, the most basic unit of operation. Its specific structure can be illustrated as follows: Figure 4 As shown. The first operation unit 18 optionally has one or more inputs 11, 12 and an output 23. These interfaces are all displayed to the user through the user interface UI_3 for connection or configuration operations. Further, step S-2) of the configuration method in some embodiments also includes: in response to the user's drag operation on multiple operation units operat_3, presenting multiple operation units operat_3 in the configuration area config_3; and in response to the user's connection operation on multiple operation units operat_3 in the configuration area config_3, connecting the first output interface 23 to the input interface (not marked in the figure) of the adjacent second operation unit in the processing flow process_3, thereby forming the entire processing flow process_3 step by step. The processing flow process_3 can be regarded as a (configurable) rule engine. The image data DS10_1 or its recognition result input to this rule engine may need to meet certain requirements. For example, if the recognition result of the image data DS10_1 has the following attributes, it is more conducive to the construction of the processing flow process_3:

[0171] A) The DS10_1 recognition results for each image can contain various attributes of each product SKU and other items (such as price tags, materials, store logos, etc.), including name, ID, brand owner, brand, series, category, series, specifications, packaging, etc. The format of the recognition results is not limited; JSON, XML, HTML, etc. are all acceptable. This helps to identify the quantity and type of different products.

[0172] For example, suppose the recognition result of the image data is: to calculate the purity of the freezer.

[0173] The attributes of the image data include: whether the sales point to which the photo belongs is a large retail store.

[0174] There might also be related questionnaire answers: Please enter whether the freezer is the first item in your input list.

[0175] Correspondingly, the final output of process_3 is: the freezer is in the first position, and the freezer's purity meets the requirements (95% for hypermarkets and 85% for other types of sales outlets).

[0176] B) The recognition results of the DS10_1 image data for each image should include the coordinates of each SKU and other items (such as price tags, materials, store signs, logos, etc.) in the image. The coordinates can be expressed as a percentage of the coordinate point in the image or the pixel position of the coordinate on the image.

[0177] C) The recognition result of the DS10_1 image data for each image should include the door, shelf, shelf section, shelf layer and its coordinates in the freezer scene, as well as its relationship with the SKU and other items (such as price tags, materials, store signs and logos). For example, the SKU is located on which door and which shelf from left to right in the image. JSON, XML, HTLM, etc. are all acceptable. The relationship can be represented by an index or a structure containing hierarchical relationships.

[0178] D) The DS10_1 recognition result for each image should contain the adjacent relationships between each SKU and other items (such as price tags, materials, store logos, etc.). For example, product A is to the left of product B, product B is above product B, and there are no products to the right of product C. Adjacency relationships can be represented using an index or a path table.

[0179] However, due to the varying recognition capabilities of different image recognition tools, the results of image recognition may not always meet all requirements. If some or even all requirements are not met, it does not affect the configurability of the architecture and processing flow of this rule engine invention itself, but it may affect the number of rules that can be superimposed on the rule engine. The less information the input parameters provide, the fewer rules can be superimposed. These situations do not constitute a limitation on the present invention; the scope of protection of the present invention should be defined by the appended claims.

[0180] Setting of the arithmetic unit

[0181] Since the final result requires multiple steps of processing corresponding to multiple operation units (operat_3) before it can be calculated, the architecture of the process_3 should preferably support the use of the calculation result of any step for the next step. This allows for the creation of any form of process_3 through user-configured operations at any granularity, ultimately yielding any desired result. To achieve this, it is preferable to design the smallest, most basic operation unit (or operation rule, operator) and strictly define the input and output types of the operation unit to ensure that it can be used as input parameters by the next operation unit.

[0182] The structure of each arithmetic unit 18 can be referenced. Figure 4 To clarify, a processing unit 18 may optionally include one or more of the following four parameters / interfaces / capabilities:

[0183] A) One or more input interfaces 11, 12: The input interfaces here include operation parameters. The final result of the processing flow process_3 can be obtained through a series of logic steps after passing through the input parameters.

[0184] B) Set parameter set_3: This is a construction parameter, which is a set of values ​​that affect the final logical operation. For example, when sorting an array of numbers, the parameter can be set to either forward or reverse order.

[0185] C) Output parameter 23: This is the final result obtained by converting the input parameters into actual output parameters through calculation by the arithmetic logic based on the configured values ​​of the set parameters.

[0186] D) Operational Logic: This refers to the logical functions that implement the process from input parameters + construction parameters to output parameters. Since each logical function requires development and implementation, each operational unit also needs to be developed. However, once implemented, it can be directly used by users through drag-and-drop interface operations. This is similar to each method or function in a Java public class library; once implemented in advance, it can be used by all Java developers. Different display checks require different combinations of operational rules for targeted implementation. For example, calculating the number of shelf spaces and finding concentrated displays correspond to different operational rules / units or combinations. Some operational units require specific parameter / attribute settings to perform the corresponding operations. For example, the operational unit used for SKU filtering needs to have corresponding filtering conditions set according to the specific requirements of the display verification / inspection.

[0187] After a processing unit has the above four parts, its structure can be as follows: Figure 4 As shown.

[0188] Optionally, in some embodiments involving setting parameter set_3 in B) above, the first arithmetic unit opera7_5 may only have a first output interface, and further includes the step of: setting the attribute / parameter set_3 of the first arithmetic unit according to the user's configuration operation, so as to apply it to the second arithmetic unit or processing flow through the first output interface. Here, the first arithmetic unit provides a configuration interface UI_6 for the user, such as... Figure 6 As shown, the human-computer interaction interface UI_6 allows users to configure the above-mentioned setting parameter set_3 of the first calculation unit. For example, the setting parameter set_3 of the first calculation unit can be the display ratio threshold of a certain SKU, which is set to 0.95 here. This causes the processing flow process_7 to compare with the value 0.95 at the stage of passing through the first calculation unit, in order to check whether the display ratio value obtained in the previous stage of the processing flow process_7 has reached the threshold of 0.95. This threshold may be one of the display rules defined by the brand for the distribution of target products.

[0189] Optionally, in this embodiment, the first operation unit opera7_5 can serve as the overall data input interface for the processing flow process_7 formed by the operation units, and can be used to import data reflecting the structured / hierarchical / layered product display for the processing flow process_7.

[0190] Optionally, in some embodiments, the data includes image data of an array of image elements, or may be as follows: Figure 11a , 11b The questionnaire data shown replaces or is used in conjunction with image data in the processing flow process_7; and multiple operation units operat_3, each representing a basic operation, or minimum operation unit, or operator, on the image element array and SKU array in the image recognition result IMR_5. Alternatively, multiple operation units operat_3 represent basic operations on the questionnaire data. Or, multiple operation units, each with operations on the questionnaire data. Figure 11a , 11b A basic operation for processing combined data of questionnaires and image element arrays.

[0191] Figure 11a , Figure 11b Two types of questionnaire data applicable in some embodiments of the present invention are shown. Figure 11a The example shown is a manual questionnaire: it is mainly used in situations where images cannot be collected normally, such as when the store does not allow taking photos and it is impossible to collect images. Figure 11bThe questionnaire shown is a "non-IR (image recognition) questionnaire": it is mainly used when IR cannot make a judgment, such as "whether the freezer is powered on" or "the placement of the freezer".

[0192] The first arithmetic unit opera7_5 with setting parameter set_3 and its overall processing flow process_7 in this embodiment can be found in [reference needed]. Figure 7 To better understand the application targeted by setting parameters set_3, the first arithmetic unit opera7_5, and the related overall processing flow process_7, the following section will combine... Figure 7 , Figure 3 , Figure 5 and Figure 1 Provide a detailed explanation. Figure 7In one embodiment of the present invention, the process of verifying whether the purity of KO soda in the freezer is greater than or equal to 95% based on image recognition results involves at least nine operation units opera7_1, opera7_2, ..., opera7_9. In response to the user's selection and dragging operation of the basic operation units within the operation unit area operaraa_3, the above nine operation units are provided in the configuration area config_3. Based on the user's connection operation between the above nine operation units, the outputs and inputs of the above nine operation units are sequentially connected to form a processing flow process_7 in the form of a directed acyclic graph. Optionally, the configuration method of the relevant embodiment further includes step S-3): importing the recognition result IMR_5 of the image data DS10_1 or questionnaire data into the input node of the entire processing flow process_7, i.e., the operation unit Opera7_1. And, in step S-4) of the configuration method: running the processing flow process_7 to process the recognition result IMR_5 of the image data DS10_1 or the questionnaire data. Through steps S-3) and S-4), based on the configuration information of each node in the data processing flow process_7 (e.g., the setting parameter set_3 of the first operation unit opera7_5) and the connections with adjacent nodes, along the directed acyclic graph of the processing flow process_7, the corresponding functions of the nine operation units in the processing flow process_7 are activated, data / parameters are transmitted, and the final processing results of the recognition result IMR_5 for the image data DS10_1 and / or the questionnaire data are presented to the user. Specifically, in the directed acyclic graph of the processing flow process_7, along the direction of the processing flow, the application scenario targeted by the processing flow process_7 is filtered / selected through the operation unit opera7_2. For example, in response to the user's configuration operation, the processing object applicable to the operation unit opera7_2 in this process is determined to be the freezer, that is, the operation unit opera7_2 is used to process the image recognition result of the freezer in this process. Here, the setting of the scenario can also be understood as the construction parameter in "B) Setting Parameters" above. Next, the operation unit opera7_2 is divided into two processing branches. In one processing branch, the operation unit opera7_8 filters the SKU list of KO soft drink in the freezer. In this embodiment, the SKU list of KO soft drink includes, for example, KO4_1 corresponding to the image area of ​​the first shelf, KO4_2 corresponding to the image area of ​​the second shelf, and KO4_3 corresponding to the image area of ​​the third shelf. Then, the operation unit opera7_9 calculates the SKU display area of ​​the KO soft drink product: 3. In the other processing branch after the operation unit opera7_2, the operation unit opera7_3 calculates the total number of display areas of the relevant freezer, for example, the result is 4.Next, the calculation results of operation units opera7_3 and opera7_9 are input to the input interface of operation unit opera7_4. Through division, the proportion of each unit is obtained, i.e., the proportion of KO soda SKU shelf space in the total shelf space of the freezer. Then, the calculation result of operation unit opera7_4 is input to the input interface of operation unit opera7_6. Simultaneously, the output of operation unit opera7_4 is also connected to the input of operation unit opera7_6. The function of operation unit opera7_6 is to compare sizes. Here, it compares the calculation result of operation unit opera7_4 with the set parameter set_3 of operation unit opera7_5. This set parameter set_3 has been set to 0.95. If the calculation result of operation unit opera7_4 is, for example, 0.75, then the shelf space proportion of KO soda in this freezer scenario is lower than the display rule defined by the brand for this type of product. Finally, the comparison results of the above shelf space ratio can be output in a visual way through the operation unit opera7_7 so that users can directly obtain this information, such as "The shelf space ratio of KO soda in this freezer is lower than the value required by the brand's display rules: 0.95".

[0193] Here, it should be understood that when executing nine computation units along the directed acyclic graph of the processing flow process_7, it is necessary to parse the rules corresponding to multiple computation units and execute them in a certain order. Optionally, topological sorting can be used to decompose the computation tasks of rules with pre-defined computational relationships. This topological sorting ensures that if computation unit A needs the output parameters of computation unit B, then computation unit B will be run first. Furthermore, since the processing flow process_7, composed of multiple computation units, is a directed acyclic graph, there is no need to worry about nested loops causing uncertainty about the order of computation units A and B.

[0194] Therefore, preferably, in some embodiments, the directed acyclic graph formed by configuring multiple computing units and their corresponding computing rules can be transformed into a task list with an execution order. Each task list records its own computing logic and the source of the input parameters, i.e., the output result of a previously executed operation. During the execution of process_7, computing tasks are executed sequentially according to the task list in the form of a directed acyclic graph composed of computing units, and the computing logic corresponding to the computing units in the computing task list is called. After the computing task of a certain computing unit is completed, its output result can be recorded so that it can be called by computing tasks corresponding to other computing units later. If it is marked as a final output item, such as the calculation result of computing unit Opera7_7, it is preferable to store it separately and return it to the business system.

[0195] In this way, the product display definition is subdivided and corresponding to basic computing units. The processing flow process_7 is constructed through these basic computing units. Thus, by analyzing the images of the product Prod_1 displayed on the shelf Shelf_1 and other display devices at the distribution point on the server SV109, it is possible to check whether the display of product Prod_1 on the shelf Shelf_1 and other display devices meets the brand owner's display requirements / display rules for these products.

[0196] As long as the basic computational units defined in the detailed specifications are relatively complete, different directed acyclic graphs can be created by splicing, connecting, and associating these computational units. This allows for the calculation of desired results for any display requirements / rules of these products from the brand. Furthermore, since the basic computational unit `operat_3` has already been implemented and provided in `operarea_3`, flexible configuration of the display rule review / verification process at server SV109 can be achieved simply by configuring the connection, without consuming a significant amount of time writing, testing, and deploying hard code. This also improves the efficiency of reviewing the image recognition result IMR_5.

[0197] The connection between multiple operation units and their corresponding operation rules can be in the form of directed acyclic graphs or directed acyclic trees, i.e., logic trees. While using logic trees may compromise usability in scenarios where the output parameters of one operation unit need to be used by multiple other operation units, it is still possible to obtain any desired result from the process_7 workflow through configuration.

[0198] Optionally, in some embodiments of the configuration method, the raw image recognition data from the terminal device is preprocessed (e.g., image recognition processing) to obtain the recognition result IMR_5 of the image data. Step S-4) further includes: in response to the operation of running the processing flow Process_7, sequentially running the functions corresponding to the nine operation units in the processing flow to process the image recognition data (or simply image data); and in step S-5) of the method in this embodiment: feeding back the processing result of the image data to the terminal device. Through this first operation unit opera7_5 with setting parameter set_3, the adaptability, configurability, and configuration efficiency of the processing flow for product display definitions are improved. Furthermore, through this processing flow Process_7, it is possible to... Figure 1The system consists of the DS10 terminal device and the SV109 server. The review results from the software on the server are quickly fed back to the terminal device. Before the adoption of the modularly configurable Process_7 workflow in the SV109 server, images / photos collected by the DS10 terminal device were uploaded to the backend, such as the SV109 server. Sometimes, these images required manual visual identification, which was inefficient, time-consuming, and consumed significant human resources. If a non-configurable, code-based software workflow were used to process the image data collected by these terminal devices, the software workflow would be unable to adapt to the frequently changing and diverse image review rules. Furthermore, the DS10 terminal device might not differentiate between scenes when collecting images, easily resulting in a large number of duplicate images that fail to accurately reflect the sales point situation.

[0199] Overall, this also leads to a long cycle in which the review results for the collected images are delivered from the server SV109 to the terminal device DS10 in this system consisting of the terminal device DS10 and the server SV109. The terminal device DS10 cannot receive the review results for the collected image data quickly or even in real time. Consequently, the sales points whose images of freezers, shelves, and floor displays are collected by the terminal device DS10 cannot make timely rectifications based on the aforementioned review results.

[0200] After applying the configuration methods of some embodiments of the present invention, and the on-demand configurable processing flow, the feedback speed of the above-mentioned audit results reports is improved, for example, from once every two weeks to twice a week, or even real-time reporting. This may enable closed-loop rectification of product placement projects that do not comply with product display rules. In addition, it can also allow the terminal device DS10 and its users to take photos according to the scenario, such as taking photos of freezers, shelves, and floor promotions in the store where the product is placed, to reduce duplicate photos.

[0201] Of course, the multiple computational units here can be understood as the breakdown of the brand's preset product display definitions / rules. Therefore, optionally, in some embodiments, if the brand's definition of its target product display is obtained through a formatted data collection table, step S-0) can be included before step S-1), where the formatted product display definition is broken down / parsed at the server or terminal device to obtain at least some of the multiple computational units. Some or all of the computational units are obtained through automatic machine parsing of the product display definition, or further with manual correction, to obtain all computational units and provide them in the available computational unit area operaraa_3. Correspondingly, when collecting the brand's display definition of the target product in display devices such as shelves / floor displays / freezers, it is necessary to use formatted, structured tables and other means as much as possible to facilitate further automated breakdown / parsing by the terminal or server.

[0202] Input / output parameter type definition

[0203] The most basic components of the process flow (process_3) are: Figure 4 The most basic operational rules and operational units shown are further defined to allow these operational units to be flexibly connected. Optionally, here, the input and output parameters of different operational units 18 in multiple operational units (operat_3) share a common set of type standards, but there are no restrictions on which specific types will be used.

[0204] Each operation unit may have one or more input parameters 11, 12. Some operation units may not have input parameters. The types of these parameters include, but are not limited to: numeric type, text type, bool type, and objects with attribute fields. For example, each SKU can be an object with attributes such as brand owner, brand, and category. Input and output parameters 11, 12 can also include arrays of various dimensions.

[0205] The computational units are associated with each other as a directed acyclic graph:

[0206] Multiple operation units (operat_3) are associated / connected, so that the calculation result of the previous operation unit can be further processed by the next operation unit.

[0207] After defining the types of inputs and outputs 11 and 12, it becomes possible to connect the computational capabilities where the output type of the upper-level unit is consistent with the input type of the lower-level unit, thus obtaining a directed acyclic graph. Here, the processing flow process_3 can optionally be configured as a directed acyclic tree, preferably a directed acyclic graph, the latter being suitable for the following situation: the output value of a certain computational unit may need to be used as input by multiple lower-level computational units.

[0208] It should be noted that the connection and parameter passing between operation units are not limited to any data structure; they can be a structure with an index, or other methods.

[0209] Creating interactive methods for directed acyclic graphs, in Figure 3 In the embodiments shown, the UI interface can be implemented through user clicks and connections, but it is not limited to UI interfaces for such operations. It can also be an input-based settings interface or any other interactive operation that can create relationships between rules. The specific method does not constitute a limitation of this application.

[0210] Output item definition

[0211] The output parameters are the output values ​​obtained after calculation by each unit. Since one of the purposes of Process_3 is to output some calculated results to the business system / user, it does not mean that all calculation results need to be output to the business system / user. Some are only intermediate values ​​in the calculation process of Process_3. Once the output parameter expected to be output to the business system / user is marked, the output parameter can be called an output item.

[0212] Optionally, each output item will be assigned a unique ID to distinguish it from other output items. This can be done by returning the output item to the business system immediately after obtaining the result for each item, or by creating a larger set of output items such as... Figure 8 After all the output items of Process_8 shown in the process flow have obtained the results, the output items are returned to the business system. Figure 8 The diagram illustrates the processing flow of a multi-output MO_8 consisting of a plurality of arithmetic units according to another embodiment of the present invention.

[0213] It should be noted that the arithmetic units and the processing flow formed by connecting the arithmetic units in the various embodiments of the present invention are merely illustrative descriptions for ease of understanding and are not intended to limit the present invention.

[0214] by Figure 8 For example, a total of six basic computing units are involved. These units are combined and connected in response to user selections and configurations, enabling automatic calculation / judgment of whether two SKUs are placed side-by-side, or even whether multiple SKUs are placed sequentially or randomly. These SKU placement requirements may originate from the definitions of the relevant product brands. Verification of these rule definitions can be achieved by importing image recognition results of product display in shelves / freezers through the processing flow described in this embodiment of the invention, and then automatically completing the verification process in response to user operations. The processing flow / rule engine, composed of multiple computing units, is dynamically built / constructed by the user based on the brand's rule definitions, by combining / connecting various related computing units. This improves the adaptability and configurability of the software-based processing flow to the diverse and ever-changing product display rule definitions from brand owners.

[0215] Figure 9 The following illustrates a processing flow (Process_9) for determining whether Red Bull and Monster Energy beverages are displayed adjacent to each other in an image, according to another embodiment of the present invention. Figure 9 , Figure 1 and Figure 5 This describes the process of running Process_9 after the user constructs it through interactive operations such as dragging, connecting, etc.

[0216] Figure 1 The embodiment shows the original image of Red Bull (hongniu_1) and Monster Energy (mozhua_1) displayed on Shelf_1. The image data was acquired by the terminal device DS10, and then uploaded to the server SV109. After image recognition, the image recognition result IMR_5 was obtained, such as... Figure 5 As shown. (Through) Figure 1 The images in the image determine that Red Bull (hongniu_1) and Monster Energy (mozhua_1) are displayed adjacent to each other on Shelf_1 in a physical store. At server SV109, the image recognition result IMR_5 is imported / input into processing flow Process_9 via computing unit Opera9_1. The output of computing unit Opera9_1 is passed to computing unit Opera9_2, where the user of server SV109 can input / select the actual scenario applicable to processing flow Process_9, such as the shelf. Next, computing units Opera9_3 and Opera9_7 can respectively perform SKU filtering and centralized display acquisition. Specifically, in the recognition results after filtering the shelf scenario, SKUs that meet the filtering rules are selected. The filtering conditions can be brand owner, brand, category, series, flavor, specifications, and SKU ID and name, such as Red Bull and Monster Energy in this embodiment. Furthermore, the centralized display of Red Bull and Monster Energy Original is obtained by using the centralized display processing units Opera9_3 and Opera9_7, respectively. The inputs of processing units Opera9_3 and Opera9_7 are connected to processing unit Opera9_4, which can obtain a set of adjacent SKUs for centralized display. Connected to processing unit Opera9_4 is processing unit Opera9_5, which can determine whether Red Bull (hongniu_1') and Monster Energy Original (mozhua_1') in the image recognition data IMR_5 are adjacent based on the adjacent SKUs obtained from processing unit Opera9_4, thereby determining whether Red Bull (hongniu_1) and Monster Energy Original (mozhua_1) are displayed adjacently on the Shelf_1 shelf in the offline store. Finally, in processing unit Opera9_6, which is the output, based on the judgment result of processing unit Opera9_5, such as... Figure 10 As shown in the image. Furthermore, in the actual offline collected raw image data DS10_1, it can be visually identified that Red Bull (hongniu_1) and Monster Energy Original (mozhua_1) are adjacent, consistent with the calculation results from Process_9.

[0217] Optionally, in some embodiments, the image element array on the identification image IMR_5 includes rows / columns of image elements that correspond to the rows of products actually displayed on the shelf Shelf_1, in a container, freezer, or floor stack.

[0218] It is understandable that the raw DS10_1 image data is generated by photographing shelves, containers, freezers, or floor stacks. The image element array in the IMR_5 image data also corresponds to the product / product Prod_1 in the shelf, container, freezer, or floor stack. For example, each image element in the image element array has location information, corresponding to the product Prod_1 in the shelf, container, freezer, or floor stack. After preprocessing such as cleaning the raw DS10_1 image data, the corresponding product SKU information can also be generated and associated with each image element, such as image element Hongniu_1'.

[0219] Among them, the operation units Opera9_3 and Opera9_7 are used to obtain centralized displays: if the input parameters, such as the SKUs of Red Bull hongniu_1' or Mozhua Original Flavor mozhua_1', are adjacent, a centralized display is formed. Finally, based on the layout of Red Bull hongniu_1' and Mozhua Original Flavor mozhua_1' on the recognition image IMR_5, multiple centralized displays may be formed.

[0220] The operation unit Opera9_7 is used to merge adjacent centralized displays: if two centralized displays for Red Bull hongniu_1' and Monster Original mozhua_1' are adjacent, they are merged into a new centralized display.

[0221] The operation unit Opera9_5 is used to determine if there is an array length greater than 0: it checks if there are any elements in the array. If there are, it returns True, indicating that there are Red Bull hongniu_1' and Monster Original flavor mozhua_1' adjacent to each other. Otherwise, it returns False, indicating that there are no Red Bull hongniu_1' and Monster Original flavor mozhua_1' adjacent to each other.

[0222] Figure 10 It shows Figure 9 The output of the process that determines whether Red Bull and Monster Energy drinks are displayed adjacently is the result of the entire process flow Process_9 output by the computing unit Opera9_6.

[0223] It should be noted that although some embodiments of the present invention use graphical interaction to configure the processing flow in the form of a directed acyclic graph, it also describes... Figure 7 , Figure 8 , Figure 9 After several directed acyclic graph (DAG) processing flows have run, how should the image recognition results input into these flows be processed? However, the spirit of this invention is not limited to this. It also includes, in the field of image recognition, using a (sequential) processing flow preferably in the form of a DAG to implement a rule engine for image data and image recognition results, or mapping / implementing a rule engine for image data and image recognition results as a DAG-form processing flow. This modular configuration method of processing flows based on basic computing units is also within the scope of protection of this invention.

[0224] As can be seen from the above embodiments, the modular configuration of the processing flow involves basic computing units. Optionally, in implementing these basic computing units, a set of methods can be defined to manage the input and output parameter types of the computing units. Each method represents a parameter of a certain type, such as a number, text, or some kind of object.

[0225] Alternatively, a set of methods can be defined to manage each computation unit or its corresponding computation rules, where each method in the set corresponds to an implementation of a computation unit. When validating input parameters and calculating output parameters, the methods of the computation unit will call the methods of the parameter types established in the first step.

[0226] Alternatively, a method can be defined to manage the structure of the directed acyclic graph. We adopted a JSON format, treating each operation unit as a node and recording its child nodes, i.e., which other operations the output of that operation unit is used for. If the processing flow is in the form of a directed acyclic graph, there may be multiple child nodes; if the processing flow is in the form of a directed acyclic logical tree, there may only be one child node.

[0227] Optionally, as shown in the accompanying drawings of some other embodiments of the present invention, users can generate JSON structures through a drag-and-drop UI interface. Therefore, a method needs to be created for the front end to automatically generate a new JSON after the computation unit or its corresponding computation function is connected.

[0228] Optionally, a trigger for a computation unit or its corresponding computation rule can be established, allowing higher-level business systems within the software to invoke and execute a specific rule. The specific rule can be identified using an ID defined for each computation unit, also known as a ruleID. That is, the JSON structure representing the processing flow of the directed acyclic graph is decomposed into a task queue using topological sorting. Simultaneously, for computation units marked as output items, their corresponding tasks are specially tagged, enabling feedback to the business system or presentation to the user.

[0229] In the processing flow, each computation unit corresponds to a computation task. In the directed acyclic graph, each task in the processing flow task queue is executed sequentially, and the output parameters after the computation is completed are recorded. If the input parameter when executing a task is the output parameter of a previous task, the result / output field value of the target task / computation unit in the task queue is retrieved.

[0230] Other embodiments provide examples of processing flows such as "determining whether two SKUs are adjacent" and "the number of display positions involved in a certain SKU". Some basic computing units are proposed, such as: the computing unit opera7_9 for calculating the display positions, the computing units opera9_3 and opera9_7 for filtering the centralized display, and the computing unit opera9_2 for application scenario filtering: selecting the application scenario targeted by the processing flow.

[0231] However, the basic arithmetic units in the embodiments of the present invention are not limited to this. In order to construct more diverse processing flows, more basic arithmetic units can be configured. The following is a more complete illustrative description of these possible arithmetic units.

[0232] Optionally, in some embodiments, the basic operations include at least one or more of the following: logical operations, mathematical operations, array operations, topological relationship operations, positional relationship operations, and corresponding product SKU-related operations on the image element array or one or more image elements therein. The topological relationship operations, positional relationship operations, and at least some of the SKU-related operations are based on the position of each image element in the image element array. Optionally, i) SKU-related operations, such as the operation unit Opera7_8 for filtering the SKU list, and ii) at least one of the logical operation unit, mathematical operation unit, and array operation unit, are generally configured upstream / upstream in the processing flow, relative to iii) the topological relationship operation unit and / or iiii) the positional relationship operation unit, such as the operation units Opera9_4 and Opera9_5 for determining whether two SKUs are displayed adjacent to each other. In other words, the topological relationship operation unit and the positional relationship operation unit need to process the SKU information of the image elements and process the image elements in the image element array in an array manner or logical manner.

[0233] Optionally, in some embodiments, SKU-related operations include at least one of the following three:

[0234] I) Filter the SKU list: Define filter criteria and / or target SKUs.

[0235] II) Grouping by conditions: Define the grouping conditions for the target SKUs and group them according to these conditions.

[0236] III) Calculate the number of facets: Calculate the number of facets for image elements with the target SKU.

[0237] Optionally, in some embodiments, the positional relationship calculation includes: filtering the centralized display of target SKUs, and / or obtaining a combination of multiple adjacent centralized displays of target SKUs; wherein obtaining a combination of multiple adjacent centralized displays of target SKUs includes: determining whether the centralized display for the first SKU and the centralized display for the second SKU are adjacent, such as... Figure 9 Examples are shown in the text.

[0238] Optionally, in some embodiments, the logical operation includes at least one of logical AND, logical OR, and logical NOT operations; and

[0239] Mathematical operations include at least one of addition, subtraction, multiplication, and division; and

[0240] Array operations include at least one of the following:

[0241] Find the length of the array.

[0242] Check if the array length is greater than 0; if it is, then the array exists.

[0243] Merge arrays: merge arrays

[0244] Array In operation: Performing the In operation on an array

[0245] Summing the values ​​in an array: Sum the values ​​in an array.

[0246] Basic array sorting: Sorts the array in ascending or descending order.

[0247] Array is all true: Check if the array is all true

[0248] Determine if any element in the array is true:

[0249] Get the maximum value in the array

[0250] Find the N maximum values ​​in an array. Find the Top N values ​​in an array. Find any N values ​​in the array.

[0251] List traversal: Replacement: Replace one characteristic of an object with another characteristic.

[0252] List filtering and traversal: filtering based on specific conditions

[0253] List sorting and traversal: Sort the list according to specified criteria.

[0254] Optionally, in some embodiments, the SKU-related operations further include at least one of the following:

[0255] Get POSM list: Get the material list for the corresponding product;

[0256] Get Attribute: Get a specific attribute of the corresponding product (e.g., a beverage, a case of drinks, etc.);

[0257] Get Price: Get the price of the corresponding product;

[0258] Is there price communication? Determine if the corresponding product has a price tag;

[0259] In addition, basic operations also include:

[0260] Dynamic scene filtering: Determine / filter a set of scenes;

[0261] Get Scene Type: Get the type of the scene.

[0262] Additionally, the computation unit can be implemented as a computation unit template comprising multiple basic computation units, involving template input parameters: computation units / nodes that act as a link and are required for template creation. This computation unit template fixes multiple computation units and their connections, forming a larger-scale computation unit that can be invoked as a whole to form a larger-scale directed acyclic graph (DAG) processing flow together with other computation units. Correspondingly, some embodiments of the method further include the step of: in response to a user's selection and save operation on at least two computation units associated with the processing flow, templateting and saving at least two computation units to the menu area operaraa_3.

[0263] Figure 12a , Figure 12b , Figure 12c , Figure 12d Four illustrative interactive interfaces are shown in some embodiments of the present invention during the template creation process.

[0264] For example, in response to user requests Figure 12a The configuration canvas area allows for the combination of multiple operation units selected by the user. This combination is then saved in the menu area for direct access and use later, eliminating the need to regenerate the combination each time it's needed. Consequently, subsequent user operations reduce the number of times operation units are selected or moved, resulting in a simpler directed acyclic graph structure for the processing logic.

[0265] Specifically, it responds to user selection of corresponding processing units / nodes via keyboard or mouse clicks; it also responds to further user configuration operations, such as right-clicking to generate and save a template. Taking the scenario of implementing a product's shelf space allocation as an example, template creation is illustrated as follows: Figure 12b As shown.

[0266] When the interface switches to Figure 12b After the template creation interface is shown, the user can drag and drop multiple calculation units into configuration area 111 and connect them to the filter SKU list 3. After the connection is complete, click the settings gear of filter SKU list 3 and set the attribute to dynamic. This template can be used to calculate the shelf share of KO soft drinks, KO juices, and KO drinking water.

[0267] Responding to user clicks Figure 12b To save the template, click the "Save Template" button, enter the template name, and then click the "Save Template" button again to complete the template creation.

[0268] Next, in the process of setting the template as a computational unit / rule, firstly, in response to the user clicking the template on the right in advanced mode to switch to the template that needs to be set as a rule, in response to the user clicking... Figure 12c In the middle, use icon 2 to set the template as the operation unit / operation rule.

[0269] Of course, the names of the operation units / operation rules can be freely chosen and are not bound to the name of the created template. Additionally, operation rules can be set to apply to all scenarios or specific scenarios, depending on the user's selection of the applicable scenario.

[0270] like Figure 12d As shown, in response to the user clicking the operation unit / operation rule 222 set on the left side of the interface, the relevant attributes are set, and the information 333 corresponding to the operation unit / operation rule can be displayed in the configuration area 444 (as shown in the figure).

[0271] "Secondary operations / Multiple operations"

[0272] Figures 13a-13c The present invention illustrates methods for performing secondary / multiple calculations and the related interactive interfaces in some embodiments. For example... Figure 13a The secondary operation interface shown includes a template area 13a_1, a first operation area 13a_2, and a second operation area 13a_3. Template area 13a_1 provides multiple templates, each corresponding to a template formed by associating multiple operation units or the output result of calculations involving associating multiple operation units. Based on these templates or output results, and combined with other operation units or combinations of operation units, further operation associations are formed, for example, for the processing flow process_3. These further association configurations can be configured through, for example... Figures 13b-13c Some interfaces are displayed, and further calculation results are output. The following will provide a detailed explanation of the secondary / multiple calculations based on the template of the calculation unit, with reference to these figures.

[0273] In response to the user selecting "+ Output Item Secondary Operation" 13a_2_1 in the first operation area 13a_2 of the image, the second operation area 13a_3 provides the setting attributes of the secondary operation for the user to configure, such as... Figure 13b As shown, some of the provided secondary operations can be logical operations for SKUs, as illustrated in the figure: 1) All true: corresponds to the logical "AND" relationship; 2) Any true: corresponds to the logical "OR" relationship; 3) Summation: corresponds to the mathematical operation "addition"; 4) Proportion: corresponds to the mathematical operation "division". In response to the user's request for further calculations, these logical operations are applied to the items in the template item area 13a_2_2 of the first operation area 13a_2. These items in the template item area 13a_2_2 may be provided in the template item area 13a_2_2 in response to the user's click / drag operation in the template list in the template area 13a_1.

[0274] Specifically, for example, a secondary operation can be used to determine whether both Sprite 330 Modern Can and Fanta 330 Modern Can are available in stores, or whether either Sprite 330 Modern Can or Fanta 330 Modern Can is available in stores. The node names for these two secondary operations are named and saved in the node name editing area 13c_3_1. In response to the user's operation request, the output results of the two processing flows in the template item area 13c_2_2, or rather, the two templates: 1) "Is Sprite 330 Modern Can available in stores?" and 2) "Is Fanta 330 Modern Can available in stores?", are used for secondary operations to further associate and expand some processing flows, including process_3. Combining the above logical operations, if all are true: both SKUs, such as Sprite 330 and Fanta 330, need to be available in some stores for the output result to be successful. Any is true: Success is achieved if either of the two SKUs mentioned above is available in some stores.

[0275] Furthermore, in response to the user's operation of "+Add Output Item" 13c_3_2, the processing flow / operation rules associated with the calculation unit for determining whether Sprite 330 and Fanta 330 are both available can be added to the output item list 13c_3_3 and saved for easy access by the user later. This improves the scalability of the processing flow.

[0276] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise; “multiple” generally includes at least two. It should be understood that the term “and / or” as used herein is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character “ / ” in this document generally indicates that the preceding and following related objects are in an “or” relationship.

[0277] While the foregoing has been described in detail for clarity and ease of understanding, it will be readily understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the invention. For example, many of the components and devices described above may be used in various combinations.

Claims

1. A method for configuring computing units for product display, comprising: S-1) provides multiple processing units through a user interface; S-2) Based on the user's operation, associate the multiple computing units to establish the processing flow of the multiple computing units; The plurality of computing units each have an interface for input and / or output; and step S-2) further includes: in response to a user's connection operation, sequentially connecting the plurality of computing units through the interface; wherein, along the direction of the processing flow, at least a portion of the output interfaces of the plurality of computing units are connected to the input interfaces of adjacent computing units. The plurality of computing units include a first computing unit having a first output interface, and step S-2) further includes: in response to a user's connection operation, connecting the first output interface to the input interface of a second computing unit among the plurality of computing units; The first computing unit further includes: setting the attributes / parameters of the first computing unit according to the user's configuration operation, so as to apply them to the second computing unit or the processing flow through the first output interface; The first computing unit can be used to import data reflecting the structured / hierarchical / layered product display for the processing flow. The data includes image recognition data of image element arrays or questionnaire data; The plurality of computational units are respectively a basic operation performed on the image element array and / or questionnaire data; and The first computing unit can also be configured to import attributes of the image recognition data or the questionnaire data for the processing flow. The image element array includes rows / columns of image elements, which correspond to the rows of products displayed on shelves, containers, freezers, or floor stacks. The basic operations include at least one or more of the following: logical operations, mathematical operations, array operations, topological relationship operations, positional relationship operations, and corresponding product SKU-related operations on the image element array or one or more image elements therein; the topological relationship operations, the positional relationship operations, and at least some of the SKU-related operations are based on the position of each image element in the image element array. Among them, at least one of i) SKU-related operations, ii) the logical operations, the mathematical operations, and array operations is located at the upper level / upstream in the processing flow relative to iii) topological relationship operations and / or iiii) positional relationship operations; The SKU-related operations include at least one of the following three: I) Filter the SKU list: Define filter criteria and / or target SKUs. II) Grouping by conditions: Define grouping conditions and group according to those conditions. III) Calculate the number of display spaces: Calculate the number of display spaces for the target SKU; The positional relationship calculation includes: filtering the centralized display of the target SKU, and / or obtaining a combination of multiple adjacent centralized displays of target SKUs; wherein obtaining a combination of multiple adjacent centralized displays of target SKUs includes: determining whether the centralized display for the first SKU and the centralized display for the second SKU are adjacent.

2. The configuration method according to claim 1, wherein, The logical operations include at least one of logical AND, logical OR, and logical NOT operations; and The mathematical operations include at least one of addition, subtraction, multiplication, and division; and The array operations include at least one of the following: Find the length of the array. Check if the array length is greater than 0. Merge arrays: merge arrays Summing the values ​​in an array: Sum the values ​​in an array. Basic array sorting: Sorts the array in ascending or descending order. Array is all true: Check if the array is all true Determine if any element in the array is true: Get the maximum value in the array Find the N maximum values ​​in an array; find the Top N values ​​in an array; find any value in an array. List traversal: Replacement: Replaces one characteristic of an object with another characteristic. List filtering and traversal: filtering based on specific conditions List sorting and traversal: Sorts the list according to specified criteria.

3. The configuration method according to claim 1, wherein, The user interface includes a menu area and a configuration area; Step S-1) further includes: The plurality of processing units are categorized and provided in the menu area; Based on the user's drag / selection operation, the corresponding calculation unit in the menu area is provided in the configuration area of ​​the interface; Step S-2) further includes: in response to the user's connection operation, establishing a connection between the output interface and the input interface of the corresponding arithmetic unit to allow the transmission of the corresponding parameters.

4. The configuration method according to claim 3 further includes the step of: In response to the user's selection and save operation of at least two computational units associated in the processing flow, the at least two computational units are templated and saved.

5. The configuration method according to claim 1, wherein, The SKU-related operations also include at least one of the following: Get POSM list: Get the material list for the corresponding product; Get Attribute: Retrieves a specific attribute of the corresponding product; Get Price: Get the price of the corresponding product; Is there price communication? Determine if the corresponding product has a price tag; Furthermore, the basic operations also include: Dynamic scene filtering: Determine / filter a set of scenes; Get Scene Type: Get the type of the scene.

6. The configuration method according to any one of claims 1-5, further comprising step S-3): importing the image recognition data and / or the questionnaire data into the processing flow; and, Step S-4): Run the processing flow to process the image recognition data or questionnaire data.

7. The configuration method according to claim 6, wherein, Preprocessing the raw image data from the terminal device to obtain the image recognition data, step S-4) further includes: In response to the operation of running the processing flow, the functions corresponding to the plurality of computing units in the processing flow are sequentially executed to process the image recognition data; and The method further includes step S-5): The processing results of the image recognition data are fed back to the terminal device.

8. The configuration method according to claim 6, wherein, The processing flow forms a directed acyclic graph.

9. The configuration method according to claim 1 may further include step S-0) before step S-1) disassembling / parses the formatted definition data for the product display to obtain at least a portion of the plurality of computing units.

10. An information processing apparatus for use in an electronic device having one or more processors, comprising: Apparatus for performing any one of the methods according to claims 1 to 9.

11. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by an electronic device having a display and a touch-sensitive surface, cause the device to perform any one of the methods according to claims 1 to 9.

12. An electronic device, comprising: monitor; One or more processors; Memory; and One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1 to 9.

13. A computer-readable storage medium storing one or more programs, said one or more programs comprising instructions that, when executed by an electronic device having a display, cause the device to perform any one of the methods according to claims 1 to 9.

Images