A method for calibrating a merchandise display location based on a vision array base

Abstract

The application relates to the technical fields of commodity display management and machine vision calibration, and discloses a commodity display shelf calibration method based on a vision array base, which comprises the following steps: collecting an image of an intelligent shoe support in a non-light-emitting state to generate an environment reference image and dividing an initial image area; controlling a target intelligent shoe support to emit light according to a preset light-emitting state sequence, determining a candidate image area according to the brightness change time sequence of the initial image area; respectively acquiring a target optical response of the target intelligent shoe support and an optical crosstalk response of a candidate interference node, generating a target source-specific vision mask through pixel-level comparison, and establishing an actual optical crosstalk relationship; controlling the light-emitting time sequence of subsequent calibration according to the actual optical crosstalk relationship, associating the logical ID of the intelligent shoe support, the vision mask and the shelf space identification to form spatial mapping data. The method can reduce the influence of optical crosstalk of adjacent shelves on the judgment of the vision area, and improve the reliability of the mapping relationship between the intelligent shoe support, the vision detection area and the display shelf.

Description

Technical Field

[0001] This invention relates to the field of merchandise display management and machine vision calibration technology, specifically a method for calibrating merchandise display locations based on a vision array base. Background Technology

[0002] In the display management of goods such as footwear, multiple storage locations are typically set up on display shelves to hold the products. RFID readers are used to acquire product tag information to identify the product in each location. To further monitor the placement status of the products, cameras can be used to capture images of the display shelves, and the detected areas in the images can be correlated with the actual storage locations on the shelves. This allows for storage location status management in conjunction with product identification information.

[0003] When using a camera device to visually inspect multiple adjacent storage locations, it is necessary to determine the corresponding area of ​​each storage location in the acquired image. Because the storage locations on the display rack are densely arranged, and direct, reflected, or scattered light from the light-emitting devices may enter the image area corresponding to adjacent storage locations, determining the visual inspection area solely based on preset storage location boundaries or a single light emission response is easily affected by the optical responses of adjacent storage locations. This makes it difficult for some pixels in the inspection area to accurately reflect the state of the corresponding storage location itself. Furthermore, the optical influences experienced by different storage locations may have directional differences, and fixed adjacency relationships are insufficient to directly characterize the optical crosstalk in the actual image.

[0004] Furthermore, after establishing the correspondence between storage locations and image areas, factors such as product changes, changes in display status, and changes in on-site optical conditions may cause deviations between the original visual inspection area and the actual inspection status. If all storage locations are recalibrated when anomalies occur, it will increase the amount of image acquisition and data processing during the calibration process.

[0005] Therefore, this invention proposes a method for calibrating merchandise display locations based on a visual array base to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method for calibrating merchandise display locations based on a visual array base. This method solves the problem that optical crosstalk between adjacent merchandise display locations affects the accurate determination of the visual detection area during the existing merchandise display location calibration process, leading to insufficient reliability of the location space mapping.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for calibrating merchandise display locations based on a visual array base, comprising: Images of the display rack are collected when the controllable light-emitting units of each smart shoe rack are in a non-light-emitting state to obtain an environmental reference image. The environmental reference image is divided into multiple initial image regions, and a correspondence between the initial image regions and the display rack shelf space identifiers is established. The target smart shoe with a known control logic ID switches the light emission state according to a preset light emission state control sequence, and the candidate image region is determined based on the matching result of the brightness change time sequence of each initial image region and the control sequence. The target smart shoe is controlled to enter a calibration light-emitting state. The calibration light-emitting image of the target smart shoe is compared with the environmental reference image in the candidate image area to obtain the target optical response distribution. Then, the target smart shoe is controlled to return to a non-light-emitting state. The candidate interference node set is determined according to the physical topology of the display rack. Each candidate interference node is controlled to enter a calibration light-emitting state in sequence. The optical crosstalk response generated by each candidate interference node in the candidate image area is obtained by image difference. The target optical response distribution and each optical crosstalk response are compared at the pixel level to generate a target source-specific visual mask and establish the actual optical crosstalk relationship between candidate interference nodes and the target smart shoe. Based on the actual optical crosstalk relationship, the calibration emission timing of the subsequent smart shoe racks to be calibrated is controlled, and the logical ID of the target smart shoe rack, the target source-specific visual mask and the display rack space identifier are associated and stored to generate spatial mapping data.

[0008] Preferably, determining the candidate image region includes: Image frames are continuously acquired during the switching of the light-emitting state of the target smart shoe sole according to the preset light-emitting state control sequence; Calculate the average brightness of each initial image region in each image frame, and form a sequence of region brightness changes according to the time order of the image frames; A light emission state sequence corresponding to each image frame is formed according to the preset light emission state control sequence; Normalized correlation calculations are performed on the brightness change sequence of each region and the luminous state sequence, and the initial image region with the largest correlation coefficient is determined as the candidate image region.

[0009] Preferably, the generation of the target source-specific visual mask includes: For a pixel in the candidate image region, if the target optical response of the pixel is greater than the minimum effective response threshold, and the maximum value of the optical crosstalk response generated by each node in the candidate interference node set at the pixel is less than the product of the target optical response and the crosstalk allowable ratio coefficient, the pixel is retained, and the target source specificity visual mask is generated based on the retained pixel. The retention conditions for the pixels are expressed as follows: ; In the formula, A represents the target smart shoe support; The target source-specific visual mask corresponding to the target smart shoe A; Candidate image regions for target smart shoe A; Let A be the set of candidate interference nodes corresponding to the target smart shoe A; Target smart shoe A in pixels The target optical response generated at the location; Candidate interference nodes Optical crosstalk response generated at the same pixel location; Indicates the minimum effective response threshold; This represents the allowable crosstalk scaling factor, and .

[0010] Preferably, establishing the actual optical crosstalk relationship includes: Pixels whose target optical response is greater than the minimum effective response threshold are defined as effective response pixels; For any candidate interference node in the candidate interference node set, when the optical crosstalk response at at least one effective response pixel is greater than or equal to the product of the target optical response at that effective response pixel and the crosstalk allowable ratio coefficient, it is determined that the candidate interference node has actual optical crosstalk to the target smart shoe. Establish the actual optical crosstalk relationship from the candidate interference node to the target smart shoe, and record the logical ID of the candidate interference node as the interference source node and the logical ID of the target smart shoe as the affected node.

[0011] Preferably, the calibration emission timing of subsequent smart shoe supports to be calibrated is controlled according to the actual optical crosstalk relationship, including: When allocating a calibration emission time period for the current smart shoe to be calibrated, the node that is to emit light concurrently with the current smart shoe to be calibrated is determined as the node to emit light concurrently, and the optical crosstalk direction of the node to emit light concurrently is queried. For any of the proposed concurrent light-emitting nodes, when the optical crosstalk direction of the proposed concurrent light-emitting node pointing to the current smart shoe to be calibrated has established an actual optical crosstalk relationship, or when the optical crosstalk direction has not yet been determined, the current smart shoe to be calibrated and the proposed concurrent light-emitting node are assigned to different calibration light-emitting time periods. When the optical crosstalk direction of each proposed concurrent light-emitting node pointing to the current smart shoe to be calibrated has been determined and no actual optical crosstalk relationship has been established, the current smart shoe to be calibrated and each proposed concurrent light-emitting node are allowed to emit light in the same calibration light-emitting time period.

[0012] Preferably, after generating the spatial mapping data, the method further includes: The RFID tag reading status is detected by the RFID reading component set on the target smart shoe rack; when the RFID tag reading status changes from not reading the tag to reading the tag and obtaining the product EPC information, the spatial mapping data is queried according to the logical ID of the target smart shoe rack to obtain the corresponding target source-specific visual mask and display rack space identifier. Obtain a visual reference image before the RFID tag reading state changes and an image to be detected after the state changes. Compare the visual reference image and the image to be detected within the effective visual detection area defined by the target source-specific visual mask to determine whether an image state change has occurred. The display rack space identifier corresponding to the product EPC is determined based on the pre-stored correspondence between the product EPC and the display rack space identifier; when the display rack space identifier corresponding to the product EPC is consistent with the display rack space identifier corresponding to the target smart shoe rack, and the image state change is detected, a verification result of product and space consistency is generated.

[0013] Preferably, the method further includes a local recalibration step: When the display rack space identifier corresponding to the product EPC is consistent with the display rack space identifier corresponding to the target smart shoe rack, and no change in the image state is detected within the effective visual detection area, it is determined that the RFID reading result is inconsistent with the visual detection result, and verification anomaly information containing the logical ID of the target smart shoe rack is generated. Based on the logical ID of the target smart shoe, query the actual optical crosstalk relationship and extract the interference source node with the target smart shoe as the affected node; The target smart shoe and the interference source node are combined to form a set of nodes to be recalibrated, and the optical response acquisition, target source-specific visual mask generation and actual optical crosstalk relationship determination are re-executed for the set of nodes to be recalibrated.

[0014] Preferably, the local recalibration step includes: The controllable light-emitting units in the set of nodes to be recalibrated are all kept in a non-light-emitting state. Under the condition that the current product placement state remains unchanged and the image acquisition parameters of the camera device are consistent during this round of recalibration, the current reference image of the candidate image area of ​​the target smart shoe is acquired. Using the target smart shoe as the current test node, enter the calibration light emission state, and re-obtain the target optical response distribution based on the difference between the calibration light emission image of the target smart shoe and the current reference image; Each interference source node is used as the current test node to enter the calibration emission state in sequence, and the optical crosstalk response is obtained again based on the difference between the calibration emission image of each interference source node and the current reference image. Based on the re-obtained target optical response distribution and optical crosstalk response, the target source-specific visual mask is regenerated and the actual optical crosstalk relationship is re-determined. The record corresponding to the target smart shoe in the spatial mapping data is updated with the regenerated target source-specific visual mask, and the relationship record between the interference source node and the target smart shoe in the actual optical crosstalk relationship table is updated.

[0015] Preferably, the temporal correspondence between the luminescence state and the image frame is established in any of the following ways: Based on a unified time reference or a pre-corrected clock deviation, a control timestamp is recorded when a light emission state control command is issued, and a camera frame timestamp is recorded when an image frame is generated. The light emission state corresponding to each image frame is determined based on the control timestamp and the camera frame timestamp. Alternatively, when controlling the controllable light-emitting unit to perform light-emitting state switching, a synchronous trigger signal is output to the camera device, so that the camera device responds to the synchronous trigger signal to acquire an image, thereby establishing a time correspondence between the light-emitting state switching time and the image acquisition time.

[0016] Preferably, the set of candidate interference nodes is determined based on the physical topology of the display rack, including: Read the pre-stored static physical topology table, which records the logical ID of each smart shoe and its relative spatial arrangement on the display shelf; Based on the row and column arrangement of the display rack positions, the smart shoe racks located above, below, to the left, and to the right of the target smart shoe rack are identified as directly physically adjacent nodes, and the diagonally adjacent nodes or adjacent layer nodes determined according to the existing position arrangement relationship are identified as nearest neighbor nodes. The directly physically adjacent nodes and the nearest neighbor nodes are jointly determined as the candidate interference node set.

[0017] This invention provides a method for locating merchandise display positions based on a visual array base. It offers the following advantages: 1. This invention establishes a correspondence between the display rack image area and the shelf space identifier, obtains the target optical response by combining the controlled emission state of the target smart shoe rack, and further obtains the optical crosstalk response formed by candidate interference nodes in the target area. Based on this, a target source-specific visual mask is generated and an actual optical crosstalk relationship is established. Therefore, the target emission response and the optical influence of surrounding nodes can be considered simultaneously during the shelf location calibration process, making the correspondence between the smart shoe rack logical ID, visual detection area, and display rack shelf location clearer. Furthermore, the obtained crosstalk relationship can be used to constrain the emission timing of subsequent calibration.

[0018] 2. This invention compares the optical response of the target with the optical crosstalk response of the candidate interference nodes within the candidate image region, and filters effective pixels according to the target response intensity and the allowable crosstalk ratio. This reduces the influence of factors such as light emission from adjacent storage locations, reflection from the surface of display racks, or reflection from the surface of goods on the target visual area, making the generated target source-specific visual mask more suitable as the image range for subsequent storage location status detection.

[0019] 3. After completing the location calibration, this invention can trigger status verification of the corresponding visual area by utilizing changes in the RFID reading status of the smart shoe rack. Furthermore, when the RFID reading result differs from the visual detection result, the range of nodes to be recalibrated is limited based on the established actual optical crosstalk relationship. This processing method can reuse the aforementioned calibration results to verify the consistency between goods and locations, and reduces the need for repeated calibration processing of irrelevant nodes when anomalies occur. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the system architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating step S200 in an embodiment of the present invention; Figure 4 This is a flowchart illustrating step S300 in an embodiment of the present invention; Figure 5 This is a flowchart illustrating step S400 in an embodiment of the present invention; Figure 6 This is a flowchart illustrating step S500 in an embodiment of the present invention.

[0021] Among them, 100 is a smart shoe sole; 110 is a controllable light-emitting unit; 120 is an RFID reading component; 130 is a microcontroller; 200 is a camera device; and 300 is a back-end server. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Reference Figure 1 This invention provides a merchandise display location marking system based on a visual array base, including multiple smart shoe holders 100 deployed in an array on a display shelf, a camera device 200 for capturing images of the display shelf, and a backend server 300 communicatively connected to the smart shoe holders 100 and the camera device 200 respectively. The visual array base refers to image space reference data composed of multiple initial image areas in the display shelf and the mapping relationship between each initial image area and the corresponding display shelf location space identifier.

[0024] Each smart shoe holder 100 has a unique logical ID. In a specific implementation, the smart shoe holder 100 includes a microcontroller 130, and a controllable light-emitting unit 110 and an RFID reading component 120, both connected to the microcontroller 130. The controllable light-emitting unit 110 changes its light-emitting state in response to a light-emitting control command issued by the backend server 300; in this embodiment, the controllable light-emitting unit 110 may be an LED light-emitting device controlled by the microcontroller 130. The controllable light-emitting unit 110 is positioned on or adjacent to the product-carrying area of ​​the smart shoe holder 100, ensuring that its generated target optical response is at least partially located within the area where product placement can cause image state changes. The RFID reading component 120 reads the EPC information of the product placed on the smart shoe holder 100 and reports RFID tag reading state change events to the backend server 300. In this embodiment, the RFID reading component 120 is disposed on the corresponding smart shoe rack 100, and its RFID tag reading area corresponds to the product carrying area of ​​the smart shoe rack 100, so that the read product EPC information can establish a corresponding relationship with the logical ID of the corresponding smart shoe rack 100.

[0025] During the calibration process and daily monitoring, the installation position of the camera device 200 relative to the display rack and the field of view for image acquisition remain unchanged, so that the same location on the display rack corresponds to a basically consistent image coordinate area in the continuously acquired images.

[0026] The backend server 300 serves as the system's data processing and control unit. Its software logic architecture includes a communication control module, an image processing module, a calibration and scheduling module, a status verification module, and a data storage module.

[0027] The calibration and scheduling module generates light-emitting control commands for different smart shoe inserts 100 and controls the controllable light-emitting units 110 of the corresponding smart shoe inserts 100 to switch their light-emitting states via the communication control module. The camera device 200 captures image data formed by the display rack under different controlled light-emitting states and transmits continuous image frames to the backend server 300. The image processing module performs image difference, pixel comparison, and visual mask generation processing on the captured image data to obtain the target source-specific visual mask corresponding to each smart shoe insert, and determines the actual optical crosstalk relationship between nodes based on the optical response of adjacent or near-adjacent smart shoe inserts to the target image area. The target source-specific visual mask and the actual optical crosstalk relationship are stored in the data storage module, and the actual optical crosstalk relationship is simultaneously fed back to the calibration and scheduling module to control the calibration light-emitting sequence of subsequent smart shoe inserts.

[0028] After detecting a change in the RFID tag's reading status, the RFID reading component 120 reports the logical ID of the corresponding smart shoe rack 100 and the product EPC information to the backend server 300 via the communication control module. The status verification module, based on the logical ID, retrieves the corresponding target source-specific visual mask and display shelf location identifier from the data storage module, and calls the image processing module to perform image status detection on the corresponding valid visual detection area, thereby verifying the consistency between the product identity and the display shelf location identifier.

[0029] Reference Figure 2 The present invention also provides a method for calibrating merchandise display locations based on a visual array base, comprising the following steps.

[0030] Step S100: Establish image region mapping for the display rack and determine candidate image regions for the target smart shoe rack.

[0031] When all controllable light-emitting units 110 of the smart shoe racks 100 are in a non-light-emitting state, a global image of the display rack is acquired to generate an environmental baseline image. Based on the physical grid of the display rack or the preset shelf space boundary, the global image is divided into multiple initial image regions, and a mapping relationship is established between each initial image region and the corresponding shelf space identifier. Subsequently, a preset light-emitting state control sequence is sent to the target smart shoe rack with a known logical ID, and continuous images are acquired simultaneously. Based on the correspondence between the brightness change sequence of each initial image region and the light-emitting state control sequence, the candidate image region corresponding to the target smart shoe rack is determined.

[0032] Step S200: Obtain the optical response of the target node itself and the optical crosstalk response of the candidate interfering node respectively.

[0033] The target smart shoe is controlled to enter a calibration illumination state, while other smart shoe racks that need to be isolated from the target smart shoe rack according to the current calibration sequence remain in a non-illuminating state. Images of the target smart shoe rack in its illuminated state are acquired and differentially processed with an environmental reference image to obtain the target optical response distribution of the target smart shoe rack within the candidate image area. Subsequently, the target smart shoe rack is controlled to return to its non-illuminating state, and a set of candidate interference nodes is determined based on the physical topology of the display rack. Nodes in the candidate interference node set are then sequentially controlled to enter the calibration illumination state as the current test nodes, obtaining the optical crosstalk response generated by each candidate interference node within the candidate image area of ​​the target smart shoe rack.

[0034] Step S300: Generate a target source-specific visual mask and establish the actual optical crosstalk relationship.

[0035] Within the candidate image region of the target smart shoe, the optical response distribution of the target and the optical crosstalk response corresponding to each candidate interference node are compared at the pixel level. The image region that has an effective optical response when the target smart shoe emits light and does not produce an optical crosstalk response sufficient to cause source confusion when the candidate interference node emits light is selected as the effective visual detection region, and a target source-specific visual mask is generated.

[0036] Simultaneously, based on the actual optical crosstalk generated by each candidate interference node in the candidate image area of ​​the target smart shoe, it is determined whether there is actual optical crosstalk between the corresponding nodes; for candidate interference nodes that meet the preset interference judgment conditions, an actual optical crosstalk relationship is established from the candidate interference node to the target smart shoe.

[0037] Step S400: Control subsequent calibration and establish spatial mapping based on actual optical crosstalk relationship.

[0038] The backend server 300 uses the obtained actual optical crosstalk relationship as feedback conditions for subsequent smart shoe upper calibration timing control. For smart shoe uppers that have completed optical crosstalk relationship evaluation and have actual optical crosstalk relationship, they are controlled to be in different calibration emission time periods during subsequent calibration processes; for smart shoe uppers that have completed optical crosstalk relationship evaluation but have not established actual optical crosstalk relationship, emission control can be performed according to the predetermined calibration timing sequence.

[0039] As each smart shoe rack completes its calibration in turn, the backend server 300 gradually updates the actual optical crosstalk relationship. After completing the calibration of the corresponding smart shoe rack, it associates and stores the smart shoe rack's logical ID, the target source-specific visual mask, and the corresponding display rack space identifier to form spatial mapping data.

[0040] Step S500: Perform product status verification and partial recalibration based on the status change event read from the RFID tag.

[0041] In normal operation, when the RFID reading status of the target smart shoe rack changes and the product EPC information is read, the backend server 300 calls the corresponding target source-specific visual mask and display rack space identifier based on the target smart shoe rack's logical ID, and detects the image status change corresponding to the product placement within the effective visual detection area defined by the visual mask. Based on the preset correspondence between the product EPC obtained from RFID reading, the product identity, and the display rack space identifier, as well as the visual detection results, the consistency between the current product identity and the corresponding display rack space identifier is verified.

[0042] When the change in the RFID tag reading status is inconsistent with the change in the image status within the corresponding effective visual detection area, the backend server 300 determines the set of nodes to be recalibrated related to the abnormal target smart shoe based on the established actual optical crosstalk relationship. It then re-executes the optical response acquisition, source identification, and actual optical crosstalk relationship update process only for the set of nodes to be recalibrated, and updates the target source-specific visual mask, spatial mapping data, and actual optical crosstalk relationship of the target smart shoe accordingly.

[0043] To further illustrate the data processing process and specific implementation of the technical solution of the present invention, the structural relationships, data processing process and judgment conditions involved in each of the above steps are described in detail below.

[0044] To facilitate the explanation of node selection, timing correspondence, image response discrimination, and spatial mapping relationships in the subsequent calibration process, the following explains the data pre-stored on the backend server 300, the timing correspondence between light emission control and image acquisition, and the basic parameters used in image processing.

[0045] The data storage module of the backend server 300 pre-stores a static physical topology table of the smart shoe rack array. This static physical topology table records the logical ID of each smart shoe rack 100 on the display shelf and the relative spatial arrangement position of the corresponding smart shoe rack 100 on the display shelf, and records the direct physical adjacent nodes and nearest neighbor nodes corresponding to each smart shoe rack 100 using an adjacency table.

[0046] In this embodiment, directly physically adjacent nodes are determined based on the row and column arrangement of the display shelf positions, including smart shoe racks located above, below, to the left, and to the right of the target smart shoe rack; nearest neighbor nodes are diagonally adjacent nodes or adjacent layer nodes determined based on the existing shelf position arrangement. The backend server 300 directly constructs a set of candidate interference nodes corresponding to the target smart shoe rack according to the above adjacency relationship, without the need to set a separate spatial distance threshold.

[0047] When the background server 300 performs calibration scheduling, it reads the static physical topology table to determine the physical adjacency level corresponding to the target smart shoe, and obtains a set of candidate interference nodes accordingly, which serves as the physical adjacency basis for subsequent optical response extraction and calibration of light emission control.

[0048] During the light emission calibration process of the smart shoe rack 100, the backend server 300 establishes a time correspondence between the light emission state control timing and the camera image acquisition timing. The calibration scheduling module sends a light emission state switching control command to the smart shoe rack 100 through the communication control module, driving the microcontroller 130 to change the light emission state of the controllable light emission unit 110; the camera device 200 acquires continuous image frames of the display rack under the corresponding light emission state. The backend server 300 determines the controlled light emission state corresponding to each image frame based on the correspondence between the light emission state switching time and the image frame acquisition time.

[0049] In one implementation, a system timestamp is used for time alignment. The backend server 300 and the camera device 200 use a unified time base, or the clock deviation between them is pre-corrected. The calibration and scheduling module records the control timestamp when issuing the light emission state switching control command, and the camera device 200 records the camera frame timestamp when generating an image frame. The image processing module compares the control timestamp and the camera frame timestamp on the same timeline and determines the corresponding light emission state based on the acquisition time of each image frame.

[0050] In another implementation, a hardware triggering method is used to establish the time correspondence. The microcontroller 130 is connected to the external trigger interface of the camera device 200 through a synchronous trigger line. When the microcontroller 130 executes the state switching of the controllable light-emitting unit 110, it outputs a trigger signal to the camera device 200 through the synchronous trigger line. The camera device 200 responds to the trigger signal to perform image acquisition, thereby establishing a correspondence between the image acquisition time and the state switching time of the controllable light-emitting unit 110.

[0051] Based on the above time correspondence, the backend server 300 obtains the image frame sequence corresponding to each controlled luminescence state, which is used to subsequently extract the image response generated by the luminescence of the target smart shoe sole itself and the optical crosstalk response generated by the luminescence of the candidate interference nodes.

[0052] The image processing module of the backend server 300 uses a two-dimensional pixel matrix to represent the environmental reference image. The environmental reference image records the reference brightness value corresponding to each pixel coordinate when the controllable light-emitting units 110 of each smart shoe holder 100 on the display rack are all in a non-light-emitting state, and saves it in the data storage module as a differential reference image for subsequent calculation of the target optical response and the optical crosstalk response of candidate interference nodes.

[0053] To distinguish between the effective target response and the optical crosstalk response generated by candidate interference nodes, the image processing module sets a minimum effective response threshold and a crosstalk allowable ratio coefficient.

[0054] The minimum effective response threshold is used to distinguish between the effective brightness change caused by the state change of the controllable light-emitting unit 110 and the background brightness fluctuation in the non-light-emitting state. In this embodiment, multiple frames of images are continuously acquired while each controllable light-emitting unit 110 is in a non-light-emitting state. Using the environmental reference image as a reference, the absolute difference in brightness between each image frame and the environmental reference image at the corresponding pixel position is calculated, and the minimum effective response threshold is determined based on the brightness fluctuation range observed in the non-light-emitting state.

[0055] Specifically, the maximum value of the absolute difference in brightness of each pixel in the set of images can be used as the minimum effective response threshold, so that the difference results exceeding the brightness fluctuation range can be used as effective input for subsequent target response discrimination.

[0056] The crosstalk allowance ratio is used to limit the allowable ratio of the optical crosstalk response generated by a candidate interference node at the same pixel location relative to the optical response of the target smart shoe itself. This ratio is a value greater than 0 and less than 1, ensuring that the target's own optical response of the retained pixel is greater than the crosstalk response generated by the candidate interference node. In this embodiment, the crosstalk allowance ratio can be 0.5, meaning that when the crosstalk response generated by the candidate interference node at the corresponding pixel location is less than half of the target's own optical response, the pixel can continue to participate in the determination of the target source-specific visual region. The minimum effective response threshold and the crosstalk allowance ratio are used for subsequent pixel-level source discrimination.

[0057] After the backend server 300 completes the calibration of the smart shoe rack, it constructs spatial mapping data in the data storage module. The spatial mapping data includes the logical ID of the smart shoe rack, the target source-specific visual mask data, and the corresponding display rack location space identifier.

[0058] Target source-specific visual mask data is used to represent the effective visual detection area determined after optical response source discrimination, and can be represented by a set of binary pixels or region boundary contour coordinates. Display shelf location space identifiers are used to indicate the position of the corresponding smart shoe support 100 in the display shelf location arrangement.

[0059] During the candidate image region determination process, each target smart shoe rack corresponds to a display shelf location space identifier. After generating the target source-specific visual mask for the target smart shoe rack, the backend server 300 associates the logical ID of the target smart shoe rack, the corresponding display shelf location space identifier, and the target source-specific visual mask into the same spatial mapping record. Thus, the spatial mapping data establishes the correspondence between the smart shoe rack's logical identity, the effective visual detection area, and the display shelf location.

[0060] Spatial mapping data is stored in the data storage module. When an RFID tag reading status change event occurs during daily operation, the status verification module queries the spatial mapping data based on the logical ID of the corresponding smart shoe rack 100 to obtain the corresponding target source-specific visual mask and display rack location space identifier, which is used for subsequent consistency verification between the product identity and the display rack location space identifier.

[0061] Step S100 is used to establish an environmental reference image and an initial image region mapping for the display rack, and to determine the corresponding candidate image region by controlling the change in the luminous state of the target smart shoe rack. Step S100 includes steps S101 and S102, wherein step S101 is used to establish an environmental reference image and an initial image region mapping, and step S102 is used to determine the candidate image region of the target smart shoe rack by temporal matching of luminous states.

[0062] S101, Establish environmental baseline image and initial image region mapping.

[0063] After the system is started, when all the controllable light-emitting units 110 of the smart shoe soles 100 are in a non-light-emitting state, the background server 300 sends a parameter locking command to the camera device 200 to fix the exposure time, gain and white balance parameters of the camera device 200; for the camera device 200 with an adjustable aperture, its aperture parameter is fixed at the same time so that the images acquired under different light-emitting states in the subsequent calibration process use the same camera parameters.

[0064] After the acquisition parameters are locked, the camera device 200 continuously acquires images of the display rack. The image processing module selects one frame from the images acquired in the non-illuminating state as the environmental reference image and saves it to the data storage module. The remaining image frames acquired in the non-illuminating state are used to determine the aforementioned minimum effective response threshold.

[0065] The image processing module divides the environmental reference image into multiple independent initial image regions based on the actual separation boundaries of the display rack locations. The actual separation boundaries of the display rack locations are determined by the pixel boundary coordinates of each location in the global image and are pre-stored in the data storage module. For display racks with a regular row and column structure, the image regions enclosed by the corresponding pixel boundaries are sequentially determined as the initial image regions for the corresponding locations according to the row and column order of the locations within the display rack.

[0066] Each initial image region corresponds to a physical location on the display shelf. The image processing module records the region number and boundary coordinates of each initial image region, and establishes a one-to-one correspondence between the region numbers of the initial image regions and the corresponding display shelf location space identifiers according to the row and column arrangement of the display shelf locations, forming an initial region coordinate list. The environmental reference image and the initial region coordinate list are stored in the data storage module for subsequent image differencing and candidate image region determination processes.

[0067] S102, determine the candidate image region of the target smart shoe upper by matching the luminescence state temporal sequence.

[0068] After establishing the initial image region mapping, the calibration scheduling module selects the target smart shoe with a known logical ID and sends a preset light emission state control sequence to the microcontroller 130 of the target smart shoe, controlling the controllable light emission unit 110 to switch between the light emission state and the non-light emission state according to the corresponding timing sequence.

[0069] The preset illumination state control sequence consists of illumination and non-illumination states arranged in chronological order. The duration of each state is determined according to the image acquisition cycle of the camera device 200, ensuring that at least one corresponding image frame can be acquired during the duration of each illumination or non-illumination state. The background server 300 saves the illumination state control sequence and the corresponding time position of each state as a reference for subsequent image timing matching.

[0070] The camera device 200 maintains the acquisition parameters locked in step S101, synchronously acquires continuous image frames of the display stand during the switching of the target smart shoe's luminous state, and transmits the acquisition results to the image processing module. The image processing module uses the aforementioned time correspondence to determine the luminous state of the target smart shoe corresponding to each image frame, and processes each initial image region sequentially according to the initial region coordinate list.

[0071] For each initial image region, the image processing module calculates the average brightness of the pixels in that region in each image frame and forms a region brightness change sequence according to the time sequence of the image frames; at the same time, it forms a light emission state sequence corresponding to the corresponding image frame according to the preset light emission state control sequence.

[0072] The image processing module performs normalized correlation calculations on the regional brightness change sequence and the luminescence state sequence of each initial image region, using the obtained correlation coefficient to characterize the temporal matching degree between the two. In one implementation, for the ... For each initial image region, the correlation coefficient can be calculated using the following formula: ; In the formula, Indicates the first Correlation coefficients corresponding to each initial image region; Indicates the number of image frames involved in temporal matching; Indicates the first The initial image region in the first Average brightness in each image frame; Indicates the first The average value of the brightness change sequence of each initial image region; Indicates the first The luminous state values ​​corresponding to each image frame are as follows: the luminous state value is 0 for non-luminous states and 1 for luminous states. This represents the average value of the luminous state sequence. When the brightness change sequence of any initial image region has no brightness change, causing the denominator of the normalized correlation calculation to be zero, the correlation coefficient corresponding to that initial image region is determined to be 0. The image processing module determines the initial image region with the largest correlation coefficient as the candidate image region for the target smart shoe.

[0073] The identified candidate image regions are used to define the pixel processing range for the subsequent target optical response and candidate interference node optical crosstalk response.

[0074] Reference Figure 3 Step S200 is used to acquire, within the candidate image region of the target smart shoe, the target optical response generated by the light emitted by the target smart shoe itself and the optical crosstalk response generated by the light emitted by the candidate interference nodes. Step S200 includes steps S201 and S202, wherein step S201 is used to extract the target optical response of the target smart shoe, and step S202 is used to extract the optical crosstalk response of each candidate interference node pointing to the target smart shoe one by one.

[0075] S201, Extract the target optical response of the target smart shoe upper.

[0076] After determining the candidate image region of the target smart shoe, the calibration scheduling module generates a mutually exclusive light emission control command according to the current calibration timing sequence, controls the target smart shoe to enter the calibration light emission state, and controls other smart shoe in the current calibration timing sequence that need to be isolated from the target smart shoe in terms of light emission state to remain in a non-light emission state.

[0077] The camera device 200 selects an image frame corresponding to the calibrated luminous state of the target smart shoe out according to the time correspondence between the aforementioned luminous state control timing and the camera image acquisition timing, thereby obtaining the target luminous image. The image processing module acquires the target luminous image and the environmental reference image saved in step S101, and performs pixel difference calculation on the candidate image region under the same pixel coordinates to obtain the brightness change of the target smart shoe out from the non-luminous state.

[0078] Let A represent the current target smart shoe, and the target optical response is calculated according to the following formula: ; In the formula, , Represents the pixel coordinates within the candidate image region; This indicates the brightness value of the corresponding pixel when the target smart shoe sole A is in the calibrated luminous state; The reference brightness value representing the same pixel coordinates in the environmental reference image; This represents the target optical response generated by the target smart shoe A at this pixel position.

[0079] The image processing module performs the above difference calculation on each pixel in the candidate image area to form the target optical response distribution of the target smart shoe A, and saves the calculation results to the data storage module for optical response source discrimination in step S300.

[0080] S202, extract the optical crosstalk response of each candidate interference node.

[0081] After acquiring the target's optical response distribution, the calibration and scheduling module controls the target smart shoe sole A to return to a non-emitting state and reads the static physical topology table in the data storage module. Based on the aforementioned physical adjacency relationships, it obtains the candidate interference node set corresponding to the target smart shoe sole A. The candidate interference node set consists of the target smart shoe sole A's directly physical adjacent nodes and nearest neighbor nodes. Its determination method is consistent with the adjacency relationships recorded in the aforementioned static physical topology table, without setting additional spatial distance ranges or filtering parameters.

[0082] The calibration scheduling module controls each node to enter the calibration illumination state one by one according to the candidate interference node set. For any node in the candidate interference node set... control node As the current test node enters the calibration illumination state, it simultaneously controls the target smart shoe sole A and the nodes that need to be interacted with during the current test. Other smart shoe inserts that isolate the luminous state remain in a non-luminous state.

[0083] Camera device 200 selection and node By calibrating the image frames corresponding to the luminescence state, the nodes are obtained. The calibration illumination image is obtained. The image processing module performs pixel difference analysis between this image and the environmental reference image within the candidate image region to obtain the node. The brightness change generated in the candidate image area of ​​the target smart shoe A when it emits light.

[0084] To identify the source and affected nodes of optical crosstalk, the nodes are... The optical crosstalk response pointing to the target smart shoe A is expressed as: Calculate according to the following formula: ; In the formula, Indicates candidate interference nodes When in the calibrated illumination state, pixels within the candidate image area of ​​the target smart shoe A Brightness value; The reference brightness value representing the same pixel coordinates in the environmental reference image; Indicates candidate interference nodes The optical crosstalk response formed by the direct light, reflected light, or scattered light generated by the emission in the candidate image region of the target smart shoe A.

[0085] The image processing module completes the independent emission acquisition and pixel difference of each node according to the node order in the candidate interference node set, and obtains the optical crosstalk response distribution of each candidate interference node pointing to the target smart shoe A.

[0086] The target optical response distribution obtained in step S201 and the optical crosstalk response distributions obtained in step S202 are used together as inputs to step S300. They are used to compare the self-luminous response of the target smart shoe with the optical crosstalk response generated by different candidate interference nodes in the same candidate image region, and generate a target source-specific visual mask and determine the actual optical crosstalk relationship accordingly.

[0087] Reference Figure 4 Step S300 is used to determine the effective visual detection area corresponding to the target smart shoe based on the target optical response and the optical crosstalk response of each candidate interference node obtained in step S200, and to establish the actual optical crosstalk relationship between nodes based on the actual measured optical crosstalk. Step S300 includes steps S301 and S302, wherein step S301 is used to generate a target source-specific visual mask, and step S302 is used to determine and establish the actual optical crosstalk relationship between candidate interference nodes pointing to the target smart shoe.

[0088] S301, Generate a visual mask specific to the target source.

[0089] Obtain the target optical response distribution of the target smart shoe A And the optical crosstalk response of each candidate interference node pointing to the target smart shoe A Then, the image processing module performs pixel-level response comparison within the candidate image region of the target smart shoe A.

[0090] The image processing module reads the aforementioned minimum effective response threshold. Crosstalk allowance coefficient Among them, the minimum effective response threshold Used to eliminate differential response caused by background brightness fluctuations in non-emitting states; crosstalk tolerance coefficient Used to limit the allowable ratio of the optical crosstalk response generated by the candidate interfering node relative to the optical response of the target smart shoe itself.

[0091] by Representing the candidate image region of the target smart shoe A, in order to Let represent the set of candidate interference nodes corresponding to the target smart shoe A. For pixels within When the target optical response of the pixel is greater than the minimum effective response threshold And the set of candidate interference nodes The optical crosstalk response generated by each node in this pixel is less than the ratio of the target optical response to the allowable crosstalk. When multiplying by , the pixel is retained as part of the effective visual detection area.

[0092] Target source specific visual mask Represented as: ; In the formula, The target source-specific visual mask corresponding to the target smart shoe A; Candidate image regions for target smart shoe A; Let A be the set of candidate interference nodes corresponding to the target smart shoe A; Target smart shoe A in pixels The target optical response generated at the location; Candidate interference nodes Optical crosstalk response generated at the same pixel location.

[0093] According to the above judgment criteria, pixels whose target optical response does not reach the minimum effective response threshold are not included in the target source-specific visual mask; pixels whose optical crosstalk response generated by candidate interference nodes reaches or exceeds the allowable proportion are also excluded. Therefore, pixels whose source is confused due to light emission from adjacent storage locations or reflection and scattering from the surface of display racks, partitions, or merchandise are not included in subsequent visual state detection.

[0094] After the image processing module completes pixel discrimination within the candidate image region, it generates a visual mask specific to the target source. The data is then saved to the data storage module. During product status verification in step S500, the image processing module extracts image status changes within the effective visual detection area defined by the mask.

[0095] S302, determine and establish the actual optical crosstalk relationship.

[0096] During the process of generating a target source-specific visual mask, the image processing module simultaneously uses the pixel determination results of step S301 to determine whether there is actual optical crosstalk between each candidate interference node and the target smart shoe A.

[0097] Specifically, the image processing module first selects targets whose optical response is greater than the minimum effective response threshold. The pixels are determined as valid response pixels for the target smart shoe A. For the candidate interference node set... any node in Check each of the above valid response pixels one by one; when a pixel exists that makes the node... The generated optical crosstalk response reaches or exceeds the ratio of the target optical response of the pixel to the allowable crosstalk factor. When multiplying, it indicates that the node The luminous response has entered the effective response area of ​​the target smart shoe A, and has reached the level required to eliminate crosstalk of the pixel in step S301. Based on this, the node is determined. There is actual optical crosstalk to the target smart shoe A.

[0098] After confirming the existence of actual optical crosstalk, the image processing module establishes a database of candidate interference nodes in the data storage module. The actual optical crosstalk relationship pointing to the target smart shoe A is denoted as And candidate interference nodes Identify the interference source node and record it. The logical ID of the affected node A and the logical ID of the affected node A.

[0099] Actual optical crosstalk relationships are recorded separately according to the direction of interference. Node When actual optical crosstalk is generated to the target smart shoe A, establish Do they exist simultaneously? The target smart shoe A is used as a candidate interference node to the node. Determined after performing appropriate optical response testing, not based on Directly infer the inverse relationship.

[0100] The actual optical crosstalk relationship corresponding to each candidate interference node is stored in the actual optical crosstalk relationship table and fed back to the calibration scheduling module. In step S400, when the smart shoe is calibrated, the calibration scheduling module determines the nodes that need to be isolated in terms of light emission state based on the established actual optical crosstalk relationship, and adjusts the calibration light emission time period of the corresponding nodes accordingly.

[0101] After the target smart shoe A completes step S302, the crosstalk direction of each node in its candidate interference node set pointing to A is determined; among them, the direction that meets the crosstalk condition establishes an actual optical crosstalk relationship, and the direction that does not establish a relationship is considered as not detecting actual optical crosstalk in this determination.

[0102] Reference Figure 5 Step S400 is used to feed back the actual optical crosstalk relationship obtained in step S300 to the subsequent calibration process, isolate the emission time of the smart shoe rack with optical crosstalk, and establish a correspondence between the target source-specific visual mask obtained from the calibration and the corresponding smart shoe rack logical ID and display shelf space identifier. Step S400 includes steps S401 and S402, wherein step S401 is used to control the calibration emission time period of the subsequent smart shoe rack according to the actual optical crosstalk relationship, and step S402 is used to construct the spatial mapping data of the smart shoe rack.

[0103] S401 controls the subsequent calibration emission time period based on the actual optical crosstalk relationship.

[0104] During the process of optical crosstalk determination of each target smart shoe in the background server 300, the calibration scheduling module reads the list of smart shoe insulators to be calibrated, and queries the actual optical crosstalk relationship that has been determined in the data storage module when allocating subsequent calibration light emission time periods.

[0105] The calibrated light emission time period is divided according to the duration required for the aforementioned light emission state control and image acquisition process. Each calibrated light emission time period covers at least the corresponding controlled light emission state and the image acquisition process corresponding to that light emission state. The specific duration is determined based on the image acquisition cycle of the camera device 200 and the aforementioned light emission state control sequence, so that the camera device 200 can obtain the image frame corresponding to the corresponding light emission state.

[0106] For a smart shoe rack to be assigned, the calibration and scheduling module identifies other smart shoe racks that are intended to be scheduled within the same calibration illumination time period as potential concurrent illumination nodes, and queries the optical crosstalk direction of the potential concurrent illumination nodes pointing to the current smart shoe rack to be assigned. If any potential concurrent illumination node has an actual optical crosstalk relationship pointing to the current smart shoe rack to be assigned, or if the optical crosstalk direction of the potential concurrent illumination node pointing to the current smart shoe rack to be assigned has not yet been determined, then the potential concurrent illumination node and the current smart shoe rack to be assigned are scheduled in different calibration illumination time periods; only when the optical crosstalk directions of all potential concurrent illumination nodes pointing to the current smart shoe rack to be assigned have been determined, and no actual optical crosstalk relationship has been established, is the current smart shoe rack to be assigned allowed to emit light in the same calibration illumination time period as each potential concurrent illumination node.

[0107] As step S300 is executed, the backend server 300 gradually obtains new actual optical crosstalk determination results, and the calibration scheduling module updates the emission time arrangement of subsequent smart shoe soles to be calibrated accordingly. Thus, the optical crosstalk determination results obtained in step S300 participate in subsequent emission control, ensuring that smart shoe soles with actual optical crosstalk remain isolated in their emission state during image acquisition.

[0108] S402, constructing spatial mapping data for smart shoe soles.

[0109] After the target smart shoe rack completes the optical response source identification in step S300, the backend server 300 obtains the logical ID of the target smart shoe rack, the target source-specific visual mask generated in step S300, and the display shelf space identifier corresponding to the candidate image area of ​​the target smart shoe rack in step S100.

[0110] The target source-specific visual mask is used to represent the effective visual detection area corresponding to the target smart shoe rack, and can be stored in the form of a binary pixel set or region boundary contour coordinates. The backend server 300 associates the logical ID of the same target smart shoe rack, the target source-specific visual mask, and the display rack location space identifier into a spatial mapping record and saves it to the data storage module.

[0111] The backend server 300 establishes spatial mapping records one by one according to the calibration process of each smart shoe rack. After completing the calibration of the corresponding smart shoe rack in the display rack, spatial mapping data is generated by indexing the logical ID of the smart shoe rack. This spatial mapping data establishes the correspondence between the logical ID of the smart shoe rack, the effective visual detection area, and the spatial identifier of the display rack location.

[0112] When an RFID tag reading status change event occurs in step S500, the backend server 300 queries the corresponding spatial mapping record based on the smart shoe logical ID that reported the event, and obtains the corresponding target source-specific visual mask and display rack location space identifier, which are used to limit the visual detection area and verify the consistency between the product identity and the location.

[0113] Reference Figure 6 Step S500 is used to trigger the visual state detection of the corresponding storage location based on the RFID tag reading status change after the system completes the smart shoe rack calibration, and to verify the correspondence between the goods and the storage location by combining the product EPC information, the display rack storage space identification and the visual detection results.

[0114] After completing the spatial mapping data construction in step S400, the system enters the daily monitoring state. Each smart shoe rack 100 detects the RFID tag reading status in its corresponding storage location via the RFID reading component 120. When the RFID reading status of the target smart shoe rack changes from "tag not read" to "tag read" and the product EPC information is obtained, the microcontroller 130 sends the product EPC information and the logical ID of the target smart shoe rack to the backend server 300. After receiving the above information, the communication control module triggers the status verification module to perform consistency verification between the product and the storage location.

[0115] The status verification module queries the spatial mapping data established in step S400 based on the logical ID of the target smart shoe, and obtains the target source-specific visual mask and the display shelf location space identifier corresponding to the logical ID. The image processing module determines the effective visual detection area that needs to be detected based on the target source-specific visual mask, and extracts the image data corresponding to the area from the image acquired by the camera device 200.

[0116] To determine whether a change in RFID tag reading status corresponds to a change in visual status caused by the placement of the product, the image processing module retains the image frame acquired during the period when the RFID reading status of the target smart shoe is "tag not read" as a visual reference image. When the RFID reading status changes from "tag not read" to "tag read" and the product's EPC information is obtained, the image frame after the status change is acquired as the image to be detected. The image processing module performs pixel comparison between the visual reference image and the image to be detected within the same target-source-specific visual mask range.

[0117] When the camera device 200 acquires image frames before and after an RFID tag reading status change event, it maintains the same image acquisition parameters; in this embodiment, the exposure time, gain, and white balance parameters locked in step S101 are used. The image processing module compares the brightness values ​​of the same pixel positions within the target source-specific visual mask one by one, and ensures that the absolute difference in brightness between two consecutive image frames is greater than the aforementioned minimum effective response threshold. The pixels are identified as changing pixels. The minimum effective response threshold adopts the value determined above based on the background brightness fluctuation in the non-illuminated state, and is used to exclude pixel changes caused by image noise of the camera device itself and background brightness fluctuation, without setting a separate image change judgment threshold.

[0118] When there are changing pixels in the target source-specific visual mask that meet the minimum effective response condition, it is determined that the effective visual detection area has undergone image state changes before and after the RFID tag reading state changes.

[0119] The data storage module stores product identity data, which includes at least the product's EPC (Exhibition Producer Certificate) and the corresponding display shelf space identifier. The correspondence between the product EPC and the display shelf space identifier is established according to the space allocation results determined when the product is displayed, enabling a product EPC to retrieve its corresponding display shelf space identifier.

[0120] The status verification module compares the product EPC acquired by the RFID reader 120 with the display shelf space identifier corresponding to the target smart shoe rack, and performs a consistency determination based on the image status change results obtained by the image processing module. When the display shelf space identifier corresponding to the product EPC matches the display shelf space identifier of the target smart shoe rack, and an image status change is detected within the target source-specific visual mask, the product identity is determined to match the current display shelf space, and a consistency verification result between the product and the space is generated.

[0121] When the display rack space identifier corresponding to the product EPC is inconsistent with the display rack space identifier of the current target smart shoe rack, it is determined that the product and the space are inconsistent and the corresponding verification result is output; when the RFID reading component 120 has read the product EPC but no corresponding image state change is detected in the target source-specific visual mask, it is determined that the RFID reading result is inconsistent with the visual detection result and enters the subsequent local recalibration process.

[0122] As an exception handling branch in step S500 when the RFID reading result and the visual detection result are inconsistent, the backend server 300 determines the node related to the abnormal target smart shoe based on the established actual optical crosstalk relationship, and re-acquires the optical response and determines the source within this local area to update the corresponding target source specificity visual mask and actual optical crosstalk relationship.

[0123] When the RFID reading component 120 of the target smart shoe has read the product EPC information, and the image processing module does not detect the aforementioned image state change within the corresponding target source-specific visual mask, the state verification module determines that the RFID reading result is inconsistent with the visual detection result, generates verification anomaly information containing the target smart shoe logical ID, and sends the verification anomaly information to the calibration scheduling module.

[0124] When the display rack space identifier corresponding to the product EPC is inconsistent with the display rack space identifier of the current target smart shoe rack, the status verification module outputs the verification result of product and space inconsistency according to step S500; this situation indicates that the product placement position does not conform to the established space allocation relationship, and is not used as the trigger condition for the local recalibration process.

[0125] After receiving the verification anomaly information, the calibration scheduling module does not recalibrate all the smart shoe racks in the display rack, but instead queries the actual optical crosstalk relationship table saved by the data storage module.

[0126] Taking the malfunctioning smart shoe sole A as the affected node, retrieve the actual optical crosstalk relationship with the smart shoe sole A as the affected node. ,in, This indicates the interference source node that causes actual optical crosstalk to the target smart shoe A, and the interference source node is obtained. Logical ID.

[0127] The calibration scheduling module determines the target smart shoe rack A and the retrieved interference source nodes as the set of nodes to be recalibrated. Therefore, the range of nodes to be recalibrated is determined by the optical crosstalk relationship actually measured in step S300, without extending to all smart shoe racks in the display rack.

[0128] After determining the set of nodes to be recalibrated, the calibration scheduling module first controls all controllable light-emitting units in the set of nodes to be recalibrated to be in a non-light-emitting state. Under the condition that the current product placement state remains unchanged and the image acquisition parameters of the camera device 200 are consistent during this round of recalibration, the current reference image of the candidate image area of ​​the target smart shoe A is acquired, and the current reference image is used as the differential reference for this round of local recalibration.

[0129] Subsequently, the calibration scheduling module controls the target smart shoe A to enter the calibration emission state according to the mutual exclusion emission control method in step S200, and reacquires the target optical response distribution; then controls the target smart shoe A to return to the non-emission state, and sequentially controls the interference source nodes in the set of nodes to be recalibrated to enter the calibration emission state individually, and reacquires the optical crosstalk response of each interference source node pointing to the target smart shoe A.

[0130] Based on the reacquired target optical response and optical crosstalk response, the image processing module regenerates the target source-specific visual mask for the target smart shoe A according to the pixel discrimination conditions in step S300, and re-determines the actual optical crosstalk relationship between each interference source node in the set of nodes to be recalibrated and the target smart shoe A. The aforementioned minimum effective response threshold is used during the re-determination. Crosstalk allowance coefficient No separate threshold is set for local recalibration.

[0131] After the re-determination is completed, the backend server 300 replaces the original target source-specific visual mask corresponding to the logical ID of the target smart shoe rack A in the spatial mapping data with the newly generated target source-specific visual mask, and replaces the original relationship records between the same nodes in the actual optical crosstalk relationship table with the actual optical crosstalk relationship obtained from the re-determination. The logical ID of the target smart shoe rack A and the display shelf space identifier remain unchanged.

[0132] After completing the above data update, the system returns to the daily monitoring status. If a subsequent event occurs where the RFID tag reading status of the target smart shoe A changes again, the status verification module calls the updated target source-specific visual mask and actual optical crosstalk relationship, and continues to perform the consistency verification of the goods and storage location according to step S500.

[0133] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A method for locating merchandise display positions based on a visual array base, characterized in that, include: Images of the display rack are collected when the controllable light-emitting units of each smart shoe rack are in a non-light-emitting state to obtain an environmental reference image. The environmental reference image is divided into multiple initial image regions, and a correspondence between the initial image regions and the display rack shelf space identifiers is established. The target smart shoe with a known control logic ID switches the light emission state according to a preset light emission state control sequence, and the candidate image region is determined based on the matching result between the brightness change time sequence of each initial image region and the preset light emission state control sequence. The target smart shoe is controlled to enter a calibration light-emitting state. The calibration light-emitting image of the target smart shoe is compared with the environmental reference image in the candidate image area to obtain the target optical response distribution. Then, the target smart shoe is controlled to return to a non-light-emitting state. The candidate interference node set is determined according to the physical topology of the display rack. Each candidate interference node is controlled to enter a calibration light-emitting state in sequence. The optical crosstalk response generated by each candidate interference node in the candidate image area is obtained by image difference. The target optical response distribution and each optical crosstalk response are compared at the pixel level to generate a target source-specific visual mask and establish the actual optical crosstalk relationship between candidate interference nodes and the target smart shoe. Based on the actual optical crosstalk relationship, the calibration emission timing of the subsequent smart shoe racks to be calibrated is controlled, and the logical ID of the target smart shoe rack, the target source-specific visual mask and the display rack space identifier are associated and stored to generate spatial mapping data.

2. The method for calibrating merchandise display locations based on a visual array base according to claim 1, characterized in that, The process of determining candidate image regions includes: Image frames are continuously acquired during the switching of the light-emitting state of the target smart shoe sole according to a preset light-emitting state control sequence; Calculate the average brightness of each initial image region in each image frame, and form a sequence of region brightness changes according to the time order of the image frames; A light emission state sequence corresponding to each image frame is formed according to the preset light emission state control sequence; Normalized correlation calculations are performed on the brightness change sequence of each region and the luminous state sequence, and the initial image region with the largest correlation coefficient is determined as the candidate image region.

3. The method for locating merchandise display positions based on a visual array base according to claim 1, characterized in that, The generated target source-specific visual mask includes: For a pixel in the candidate image region, if the target optical response of the pixel is greater than the minimum effective response threshold, and the maximum value of the optical crosstalk response generated by each node in the candidate interference node set at the pixel is less than the product of the target optical response and the crosstalk allowable ratio coefficient, the pixel is retained, and the target source specificity visual mask is generated based on the retained pixel. The retention conditions for the pixels are expressed as follows: ； In the formula, A represents the target smart shoe support; The target source-specific visual mask corresponding to the target smart shoe A; Candidate image regions for target smart shoe A; Let A be the set of candidate interference nodes corresponding to the target smart shoe A; Target smart shoe A in pixels The target optical response generated at the location; Candidate interference nodes Optical crosstalk response generated at the same pixel location; This represents the minimum effective response threshold; This represents the crosstalk tolerance factor, and .

4. The method for locating merchandise display positions based on a visual array base according to claim 3, characterized in that, Establishing practical optical crosstalk relationships includes: Pixels whose target optical response is greater than the minimum effective response threshold are defined as effective response pixels; For any candidate interference node in the candidate interference node set, when the optical crosstalk response at at least one effective response pixel is greater than or equal to the product of the target optical response at that effective response pixel and the crosstalk allowable ratio coefficient, it is determined that the candidate interference node has actual optical crosstalk to the target smart shoe. Establish the actual optical crosstalk relationship from the candidate interference node to the target smart shoe, and record the logical ID of the candidate interference node as the interference source node and the logical ID of the target smart shoe as the affected node.

5. The method for locating merchandise display positions based on a visual array base according to claim 1, characterized in that, The calibration emission sequence of the subsequent smart shoe uppers to be calibrated is controlled based on the actual optical crosstalk relationship, including: When allocating a calibration emission time period for the current smart shoe to be calibrated, the node that is to emit light concurrently with the current smart shoe to be calibrated is determined as the node to emit light concurrently, and the optical crosstalk direction of the node to emit light concurrently is queried. For any of the proposed concurrent light-emitting nodes, when the optical crosstalk direction of the proposed concurrent light-emitting node pointing to the current smart shoe to be calibrated has established an actual optical crosstalk relationship, or when the optical crosstalk direction has not yet been determined, the current smart shoe to be calibrated and the proposed concurrent light-emitting node are assigned to different calibration light-emitting time periods. When the optical crosstalk direction of each proposed concurrent light-emitting node pointing to the current smart shoe to be calibrated has been determined and no actual optical crosstalk relationship has been established, the current smart shoe to be calibrated and each proposed concurrent light-emitting node are allowed to emit light in the same calibration light-emitting time period.

6. The method for locating merchandise display positions based on a visual array base according to claim 1, characterized in that, After generating the spatial mapping data, the process also includes: The RFID tag reading status is detected by the RFID reading component set on the target smart shoe rack; when the RFID tag reading status changes from not reading the tag to reading the tag and obtaining the product EPC information, the spatial mapping data is queried according to the logical ID of the target smart shoe rack to obtain the corresponding target source-specific visual mask and display rack space identifier. Obtain a visual reference image before the RFID tag reading state changes and an image to be detected after the state changes. Compare the visual reference image and the image to be detected within the effective visual detection area defined by the target source-specific visual mask to determine whether an image state change has occurred. The display shelf space identifier corresponding to the product EPC is determined based on the pre-stored correspondence between the product EPC and the display shelf space identifier; when the display shelf space identifier corresponding to the product EPC is consistent with the display shelf space identifier corresponding to the target smart shoe rack, and the image state change is detected, a verification result of product and space consistency is generated.

7. The method for calibrating merchandise display locations based on a visual array base according to claim 6, characterized in that, The method also includes a local recalibration step: When the display rack space identifier corresponding to the product EPC is consistent with the display rack space identifier corresponding to the target smart shoe rack, and no change in the image state is detected within the effective visual detection area, it is determined that the RFID reading result is inconsistent with the visual detection result, and verification anomaly information containing the logical ID of the target smart shoe rack is generated. Based on the logical ID of the target smart shoe, query the actual optical crosstalk relationship and extract the interference source node with the target smart shoe as the affected node; The target smart shoe and the interference source node are combined to form a set of nodes to be recalibrated, and the optical response acquisition, target source-specific visual mask generation and actual optical crosstalk relationship determination are re-executed for the set of nodes to be recalibrated.

8. The method for calibrating merchandise display locations based on a visual array base according to claim 7, characterized in that, The local recalibration step includes: Controlling all controllable light-emitting units in the set of nodes to be recalibrated to be in a non-light-emitting state, and under the condition that the current product placement state remains unchanged and the image acquisition parameters of the camera device are consistent during this round of recalibration, the current reference image of the candidate image area of ​​the target smart shoe is acquired; Using the target smart shoe as the current test node, enter the calibration light emission state, and re-obtain the target optical response distribution based on the difference between the calibration light emission image of the target smart shoe and the current reference image; Each interference source node is used as the current test node to enter the calibration emission state in sequence, and the optical crosstalk response is obtained again based on the difference between the calibration emission image of each interference source node and the current reference image. Based on the re-obtained target optical response distribution and optical crosstalk response, the target source-specific visual mask is regenerated and the actual optical crosstalk relationship is re-determined. The record corresponding to the target smart shoe in the spatial mapping data is updated with the regenerated target source-specific visual mask, and the relationship record between the interference source node and the target smart shoe in the actual optical crosstalk relationship table is updated according to the re-judgment result.

9. The method for calibrating merchandise display locations based on a visual array base according to claim 1, characterized in that, The temporal correspondence between the emission state and the image frame is established using any of the following methods: Based on a unified time reference or a pre-corrected clock deviation, a control timestamp is recorded when a light emission state control command is issued, and a camera frame timestamp is recorded when an image frame is generated. The light emission state corresponding to each image frame is determined based on the control timestamp and the camera frame timestamp. Alternatively, when controlling the controllable light-emitting unit to perform light-emitting state switching, a synchronous trigger signal is output to the camera device, so that the camera device responds to the synchronous trigger signal to acquire an image, thereby establishing a time correspondence between the light-emitting state switching time and the image acquisition time.

10. The method for locating merchandise display positions based on a visual array base according to claim 1, characterized in that, The set of candidate interference nodes is determined based on the physical topology of the display rack, including: Read the pre-stored static physical topology table, which records the logical ID of each smart shoe and its relative spatial arrangement on the display shelf; Based on the row and column arrangement of the display rack positions, the smart shoe racks located above, below, to the left, and to the right of the target smart shoe rack are identified as directly physically adjacent nodes, and the diagonally adjacent nodes or adjacent layer nodes determined according to the existing position arrangement relationship are identified as nearest neighbor nodes. The directly physically adjacent nodes and the nearest neighbor nodes are jointly determined as the candidate interference node set.

Images