Outdoor prepackaged metering automated warehouse

Abstract

The application discloses an outdoor pre-installed metering automation warehouse. In view of the problem that the bar code is partially physically shielded, a visual blind area is caused and it is difficult to avoid obstacles due to the movement and sliding of densely arranged metering assets, the application obtains the incomplete coordinates of the target bar code and the fixed physical boundary not shielded by the camera, and calculates the grid physical coordinates of the shielded object; combined with the grid boundary margin, the micro-invasion disturbance vector parameters for the shielded object are calculated; the sorting robot is controlled to perform a planar disturbance action on the shielded object according to the parameters, and the camera is controlled to continuously capture the incomplete area at a high frequency to obtain a set of time sequence images in the field of view; finally, taking the fixed physical boundary as a spatial alignment anchor point, the local feature slices newly exposed in each frame are extracted, a complete bar code reconstruction image is generated by pixel-level topological reorganization, and analysis is performed. The application can realize accurate identification of the incomplete bar code without destroying the high-density storage topology and without needing additional space.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to an outdoor prefabricated automated metering warehouse. Background Technology

[0002] With the rapid development of smart grids, outdoor prefabricated automated metering warehouses are widely used for on-site warehousing and circulation of electricity metering assets at the municipal and county levels due to their advantages such as flexible deployment and short construction period. In order to maximize the use of the limited space inside the metal compartment of the warehouse, turnover boxes are usually used to store metering assets such as single-phase meters, three-phase meters or instrument transformers in a high-density grid compact storage manner, and stacker cranes and sorting robots are used to realize unmanned operation of the whole process.

[0003] However, in actual automated circulation processes, the compactly arranged measurement assets are easily affected by the motion inertia and mechanical vibrations generated by the high-speed acceleration and deceleration of the stacker crane, making them prone to uncontrollable slight slippage or tilting within the physical grid of the turnover box. This displacement caused by motion and vibration often leads to adjacent assets pressing against and squeezing each other, resulting in the identification barcode on the surface of the target asset being partially physically obscured by adjacent assets (i.e., obstructions).

[0004] Existing warehouse visual recognition systems typically rely on a fixed global overhead view. When faced with localized physical obstructions, they often fail to extract complete barcode features, directly outputting decoding failure signals. This forces the entire automated sorting line to stop and report errors. If traditional active visual compensation methods (such as significantly tilting or moving the camera's viewpoint) are used, the camera is often unable to find an unobstructed, effective obstacle avoidance and compensation field of view due to the deep and extremely narrow grid gaps inside the turnover boxes. If the sorting mechanism is forcibly dispatched to pick up or move obstructing items significantly outside the boxes, not only will the operation process be cumbersome and severely slow down the inbound and outbound cycle, but it will also disrupt the original high-density storage topology within the turnover boxes, easily triggering a chain of mechanical interferences and collisions in the confined space.

[0005] Therefore, how to solve the visual blind spot bottleneck caused by physical obstruction and achieve accurate identification of incomplete barcodes in dense scenes without damaging the existing high-density storage structure of the turnover box and without requiring additional external obstacle avoidance operation space is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] In order to address at least one of the technical defects mentioned in the background art, the present invention aims to provide an outdoor prefabricated automated metering warehouse.

[0007] A first aspect of the present invention provides an outdoor prefabricated automated metering warehouse, including a processor, a camera, and a sorting robot, the processor being configured to perform the following operations: The camera is controlled to acquire a global overhead image of the turnover box and perform target detection. When it is determined that there is a target barcode that fails to be decoded due to physical obstruction, the coordinates of the missing area of ​​the target barcode and the fixed physical boundary that is not obstructed are extracted, and the spatial contour of the obstruction and its grid physical coordinates in the turnover box are calculated. Based on the spatial contour of the obstruction, the physical coordinates of the grid, and the preset grid boundary margin, the micro-intrusion disturbance vector parameters for the obstruction are calculated, including the flexible contact closure threshold, the horizontal micro-translation amount, and the fine-tuning rotation angle, and the spatial motion envelope formed by the horizontal micro-translation amount and the fine-tuning rotation angle is constrained within the grid boundary margin. The electric gripper of the sorting robot is controlled to move to the physical coordinates of the grid, and the obstruction is gripped with a flexible contact closure threshold. The robot performs planar disturbance actions according to the horizontal micro-translation amount and the fine-tuning rotation angle. At the same time, the camera is controlled to perform high-frequency continuous capture of the area corresponding to the coordinates of the missing area to obtain a set of temporal images of the field of view with displacement process. Using the fixed physical boundary as the spatial alignment anchor point, the newly exposed local feature slices in each frame of the visual temporal image set due to the displacement of the occluded object are extracted, and all local feature slices are reconstructed at the pixel level to generate a complete barcode reconstruction image, which is then input into the decoding engine for parsing.

[0008] A second aspect of the present invention provides an electronic device comprising a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein, when executed by the processor, the computer program instructions perform the operations as described in any of the preceding claims.

[0009] A third aspect of the present invention provides a computer-readable storage medium storing an executable computer program whose instructions, when executed by a processor, implement the operations that the processor is configured to perform as described in any of the preceding claims.

[0010] A fourth aspect of the present invention provides a computer program product comprising an executable computer program whose instructions, when executed by a processor, implement the operations that the processor is configured to perform as described in any of the preceding claims.

[0011] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects: The outdoor prefabricated automated metering warehouse provided by this invention achieves dynamic splicing and complete reconstruction of partially obscured barcodes by controlling a sorting robot to perform micro-plane perturbation on obstructed objects, and simultaneously combining high-frequency continuous capture and pixel-level topology reconstruction algorithms. This is done without disrupting the existing high-density storage structure of the turnover boxes or occupying additional obstacle avoidance space. This solution solves the technical bottlenecks of not being able to perform variable field-of-view re-capture within deep and narrow grids, and the high risk of chain interference collisions caused by forcibly removing obstructed objects. It also solves the decoding failure and shutdown error problems that are easily caused by existing global overhead shots. While ensuring the cycle time of automated production line inbound and outbound operations, it can significantly improve the success rate of visual recognition and the robustness of the system in dense warehousing scenarios. Attached Figure Description

[0012] Figure 1 These are three views of the main structure of the outdoor prefabricated automated metering warehouse disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of the main operation flow of a processor equipped with an outdoor prefabricated automated metering warehouse disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sorting robot equipped in an outdoor prefabricated automated metering warehouse disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the process of performing planar disturbance actions according to the horizontal micro-distance translation and fine-tuning rotation angle disclosed in an embodiment of the present invention. Detailed Implementation

[0013] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0014] Reference Figure 1 As shown, this embodiment provides an outdoor prefabricated automated metering warehouse, which is equipped with a processor, a global overhead camera, and a sorting robot with electric grippers (see reference). Figure 3 (As shown). Due to the extremely limited space in the metal compartments of the warehouse, metering assets such as single-phase meters, three-phase meters, or current transformers are usually stored in high-density grids within turnover boxes. In actual operation, the compactly arranged assets are prone to uncontrollable slight slippage or tilting within the physical grid of the turnover box due to the motion inertia generated by the high-speed reciprocating acceleration and deceleration of the stacker crane. This causes adjacent assets to come into contact with and be squeezed together, and the identification barcode on the surface of the target asset is partially physically obscured by adjacent assets.

[0015] To solve this technical problem without disrupting the existing high-density storage structure and without occupying additional obstacle avoidance space, refer to... Figure 2As shown, the processor in this embodiment is configured to perform the following control steps: S1, control the camera to acquire a global overhead image of the turnover box and perform target detection. When it is determined that there is a target barcode that fails to be decoded due to physical obstruction, extract the coordinates of the missing area of ​​the target barcode and the fixed physical boundary that is not obstructed, and calculate the spatial contour of the obstruction and its grid physical coordinates in the turnover box. In this embodiment, the processor controls a global overhead camera mounted directly above the turnover box to capture the current global overhead image. The camera's field of view covers the entire storage area of ​​the turnover box, ensuring that all measurement assets fall within the imaging range. After acquiring the image, the processor performs target detection on each measurement asset in the image. For example, the processor can use a target detection model based on a convolutional neural network (such as a YOLO series model or a Faster R-CNN model) to locate each measurement asset and simultaneously extract the preset barcode interest area where the identification barcode affixed to the surface of each asset is located.

[0016] Furthermore, the processor decodes each of the aforementioned preset barcode interest areas. When all preset barcode interest areas are successfully decoded, the subsequent workflow continues. If decoding of any preset barcode interest area fails, the processor does not report a decoding failure exception, but instead further confirms whether the decoding failure is caused by physical occlusion of adjacent assets. If it is determined that there is indeed physical occlusion, the barcode is marked as a target barcode, and the processor continues to extract the coordinates of its incomplete area, the fixed physical boundary of the unoccluded area, and the spatial contour and grid physical coordinates of the occluding object.

[0017] Optionally, determining the existence of a target barcode that fails to decode due to physical obstruction includes: S11, extract the preset barcode attention area corresponding to each measurement asset in the global overhead image and decode it. When the decoding of any preset barcode attention area fails, extract the edge contour features of the preset barcode attention area and its surrounding area. If decoding of a preset barcode interest area fails, the processor applies an edge detection operator to the preset barcode interest area and the surrounding area of ​​the preset pixel range (e.g., extending outward by 20 pixels).

[0018] In practical implementation, the processor can use the Canny edge detection algorithm to extract the edge contour features within the aforementioned area. Through Gaussian filtering, gradient magnitude and direction calculation, non-maximum suppression, and double-threshold edge connection, the processor can extract the physical boundaries of the asset shell and the edge information of the barcode printing area, thereby obtaining the contour feature maps of all asset physical edges and barcode edges within the preset barcode interest area and its surrounding area.

[0019] S12, based on the edge contour features, determine whether there is a physical outer contour of an adjacent asset that intrudes into the boundary of the preset barcode interest area. If so, determine that the barcode in the preset barcode interest area is a target barcode that fails to be decoded due to physical occlusion.

[0020] Based on the extracted edge contour features, the processor determines whether the physical outline of an adjacent asset intrudes into the boundary of the preset barcode interest area. Intrusion refers to the partial or complete inclusion of the physical outline segment of an adjacent asset within the boundary of the current barcode interest area.

[0021] Specifically, the processor extracts four boundary segments of the preset barcode interest area and searches within the neighborhood of each boundary segment for edge pixels belonging to the physical outline of adjacent assets. If such intrusion exists, it indicates that the barcode interest area is spatially occluded by the physical body of the adjacent asset, thus determining that the barcode within the preset barcode interest area is the target barcode whose decoding failed due to physical occlusion. If there is no intrusion from the physical outline of the adjacent asset, but the barcode decoding still fails, it is determined that the decoding failure is due to non-occlusion factors such as barcode damage, poor printing quality, or uneven lighting. In this case, a decoding failure signal is reported according to the conventional anomaly handling procedure, without triggering the subsequent micro-perturbation demasking process.

[0022] After confirming the physical obstruction of the target barcode, the spatial location of the missing area and the obstructing object is further precisely located. Optionally, the coordinates of the missing area and the fixed physical boundary of the unobstructed area of ​​the target barcode are extracted, and the spatial contour of the obstructing object and its grid physical coordinates within the turnover box are calculated, including: S13, using at least one unobstructed edge of the physical outer contour of the asset containing the target barcode as a fixed physical boundary, extracting the pixel coordinates of the fixed physical boundary in the global overhead image, and mapping it to the world coordinate system in combination with the camera's intrinsic parameters and hand-eye calibration matrix; determining the complete theoretical region of the target barcode based on the relative position of the fixed physical boundary and the preset barcode interest area, dividing the local area within the complete theoretical region that is obstructed and cannot extract effective barcode features into incomplete regions and extracting their pixel coordinates; The processor selects at least one edge within the physical outline of the metrological asset containing the target barcode, which is not obscured by any adjacent assets, as a fixed physical boundary. This fixed physical boundary must be clearly visible in the global overhead image and not overlap with adjacent assets. Since metrological assets are typically rectangular or square shells, their physical outlines usually contain at least one unobscured straight line segment, which can serve as a reliable reference benchmark.

[0023] Furthermore, the pixel-level coordinates of the fixed physical boundary in the global overhead image are extracted, and combined with the camera's intrinsic parameter matrix and the hand-eye calibration matrix, the pixel coordinates of the fixed physical boundary are mapped to three-dimensional coordinates in the world coordinate system. The camera intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients, while the hand-eye calibration matrix describes the rotation and translation transformation relationship between the camera coordinate system and the sorting robot's base coordinate system.

[0024] After completing the spatial positioning of the fixed physical boundary, the complete theoretical area that the target barcode should occupy when it is not obscured is calculated based on the relative positional relationship between the fixed physical boundary and the preset barcode interest area (this relative position has been predetermined and recorded in the system database during asset labeling). This complete theoretical area provides the theoretical complete boundary of the target barcode in space, serving as a spatial reference benchmark for determining incomplete areas.

[0025] Furthermore, barcode feature detection is performed within the complete theoretical region. Since the local area corresponding to the occluded position in this region cannot detect effective barcode features (including barcode start characters, stop characters, and bar / space edge gradients) due to the intrusion of the physical outer contour of adjacent assets, this local area is divided into incomplete regions, and the pixel coordinates of the incomplete regions in the image coordinate system are extracted.

[0026] S14, using the pixel coordinates of the incomplete area as an index, detect the physical outer contour edge of the adjacent asset that intrudes into the area, extract the contour line segment located in the area as the spatial contour of the occlusion, and determine its grid physical coordinates in the turnover box based on the centroid position of the spatial contour of the occlusion.

[0027] The processor uses the pixel coordinates of the aforementioned incomplete area as a spatial index to perform local edge detection within that area, identifying and extracting edge segments of adjacent assets that intrude into the area's physical outer contour. Specifically, the processor determines whether each pixel within the incomplete area belongs to the physical outer contour of an adjacent asset, and extracts all line segments within that area that belong to the contour of an adjacent asset as the spatial contour of the occlusion.

[0028] Furthermore, the processor calculates the geometric centroid position of the spatial outline of the obstruction and maps this centroid position to the grid coordinate system of the turnover box, thereby determining the specific grid slot where the obstruction is located and its physical grid coordinates within the turnover box. The grid coordinate system of the turnover box is pre-established, with each grid position corresponding to a fixed storage slot and having a defined physical coordinate range.

[0029] S2, based on the spatial contour of the obstruction, the physical coordinates of the grid, and the preset grid boundary margin, calculate the micro-intrusion disturbance vector parameters for the obstruction, including the flexible contact closure threshold, the horizontal micro-translation amount, and the fine-tuning rotation angle, and the spatial motion envelope formed by the horizontal micro-translation amount and the fine-tuning rotation angle is constrained within the grid boundary margin; In this embodiment, due to the extremely limited storage space within the turnover box and the small physical gaps between adjacent assets, the safe range of movement for obstructions within the turnover box's grid is extremely limited. If the disturbance exceeds this safe range, the obstruction will collide with adjacent assets, potentially even dragging them out of the grid, thereby disrupting the high-density storage topology within the turnover box. Therefore, when planning disturbance actions, the processor must strictly constrain the movement amplitude of the obstruction within the safe range defined by the grid boundary margin.

[0030] The grid boundary margin is a preset safety distance parameter, whose value is smaller than the actual physical gap between the obstruction and its adjacent assets, ensuring that the obstruction will not come into contact or collide with adjacent assets during the disturbance. The micro-intrusion disturbance vector parameters include: a flexible contact closure threshold, used to limit the clamping force of the electric gripper when holding the obstruction, ensuring reliable clamping without damaging the asset's outer shell; a horizontal micro-translation, used to limit the translation range of the obstruction in the horizontal plane; and a fine-tuning rotation angle, used to limit the rotation range of the obstruction around the vertical axis. Furthermore, the spatial motion envelope spanned by the horizontal micro-translation and the fine-tuning rotation angle (i.e., the feasible domain boundary within the two-dimensional combined space composed of the translation and rotation angle variables as independent dimensions) is strictly constrained within the parameter space range corresponding to the grid boundary margin. In other words, in the parameter space consisting of horizontal micro-translation and fine-tuning rotation angle, the set boundary of all executable parameter combinations (i.e., the value pairs of horizontal micro-translation and fine-tuning rotation angle) must not exceed the parameter safety boundary defined by the mesh boundary margin.

[0031] Optionally, based on the spatial contour of the occluder, the physical coordinates of the mesh, and a preset mesh boundary margin, the micro-intrusion perturbation vector parameters for the occluder are calculated, including: S21, calculate the theoretical amount of motion required to expose the target barcode based on the coordinates of the missing area, and use the spatial contour of the obstruction and the preset grid boundary margin to determine the maximum safe activity range under the grid physical coordinates, and limit and truncate the theoretical amount of motion to output the horizontal micro-translation amount and fine-tuning rotation angle. Based on the extracted coordinates of the missing area, the processor calculates the theoretical amount of motion required from the obstruction to fully expose the missing area of ​​the target barcode to the camera's field of view. This theoretical motion includes the theoretical translation of the obstruction along the X-axis in the horizontal plane. Theoretical translation along the Y-axis And the theoretical rotation about the vertical axis. .

[0032] Specifically, the theoretical translation is calculated as follows: Using the fixed physical boundary of the metered asset containing the target barcode as a reference, and considering the relative positional relationship between this fixed physical boundary and the pre-defined barcode interest area, the spatial distribution range of the defective area is determined. The defective area is typically located on one side of the target barcode, i.e., the side where the obstruction is located. To expose this defective area, the obstruction needs to be moved away from the target barcode, with a movement distance at least equal to the projected width of the defective area in the direction of the obstruction's movement.

[0033] Taking the X-axis direction as an example, calculate the maximum extension position of the outer boundary of the incomplete region in the X-axis direction. The innermost position of the spatial contour of the occluded object in the X-axis direction. The difference between them is added to a preset imaging margin. (This margin is used to ensure that the barcode is fully exposed in the camera's field of view rather than just being obscured.) This yields the theoretical translation along the X-axis. Similarly, the theoretical translation along the Y-axis... Calculate in the same way.

[0034] For theoretical rotation Calculate the rotation angle required to deflect the orientation of the occluding object's spatial profile away from its original direction until it no longer covers the incomplete area. Specifically, using the centroid of the occluding object's spatial profile... Using the rotation center as the pivot point, while keeping the relative distances between all points on the occluding object's spatial contour and the rotation center constant, calculate the minimum rotation angle required for the entire spatial contour of the occluding object to rotate completely out of the incomplete region. For any corner point on the occluding object's spatial contour... Its initial polar angle relative to the center of mass is The corner point at the rotation angle The new position is:

[0035] Iterate through all corner points of the occluded object's spatial outline and calculate whether each corner point still falls within the incomplete region under different rotation angles. The minimum rotation angle that ensures all corner points exit the incomplete region is the theoretical rotation amount. .

[0036] However, the theoretical amount of motion mentioned above often exceeds the safe range of movement allowed by the turnover box mesh. Therefore, the previously determined spatial contour of the obstruction and the preset mesh boundary margin are utilized. The maximum safe activity range is defined under the grid physical coordinates of the obstruction, and the specific determination method is as follows: Based on the physical boundary corresponding to the initial mesh position of the occluder, the boundary in each direction is shrunk inward by a mesh boundary margin distance. The contracted internal region is the maximum safe operating range. Specifically, for the X-axis direction, the lower limit of the maximum safe operating range... The initial left boundary coordinates of the occluding object Add mesh boundary margin value upper limit The initial right boundary coordinates of the occluding object Subtract mesh boundary margin value ,Right now:

[0037] Similarly, for the Y-axis direction, we have:

[0038] For rotation angle, the maximum safe operating range is expressed as the maximum permissible rotation angle. ,in The distance from the corner points of the occluding object's spatial outline to the mesh boundary and the mesh boundary margin. This is determined jointly. Specifically, the corner point closest to the mesh boundary in the spatial contour of the occluding object is selected, and the maximum rotation angle corresponding to the point touching the contracted mesh boundary during rotation is calculated. This maximum rotation angle is taken as the maximum allowable rotation angle. .

[0039] Furthermore, each component of the theoretical motion is compared with the previously determined maximum safe activity range, and a truncation process is applied. Specifically, for the theoretical translation in the X-axis direction... Through the limiting function Cut off, where To obscure the current X coordinate of the matter center.

[0040] when When it is in the positive direction, if Not exceeding the maximum safe activity zone's maximum movable range in the positive direction. The output horizontal micro-translation amount ;like If the movement exceeds the upper limit of the positive direction's movable range, then Cut off as .when Similarly, when the direction is negative, the cutoff is applied, and its lower limit is... The theoretical translation in the Y-axis direction Using the same amplitude limiting and truncation processing method, the output... .

[0041] For theoretical rotation ,when At that time, the output fine-tuning rotation angle ;when hour, Cut off as .

[0042] Through the above-mentioned amplitude limiting and truncation processing, the horizontal micro-translation amount output by the processor (including...) and ) and fine-tuning the rotation angle All are constrained within the maximum safe range of motion, thus ensuring that the spatial motion envelope spanned by the horizontal micro-translation and fine-tuning rotation angle does not exceed the mesh boundary margin. The defined safety boundary range of parameters.

[0043] In addition to determining the motion parameters, it is also necessary to determine the magnitude of the clamping force that the electric gripper should apply when gripping the obstruction.

[0044] S22, determine the asset type of the obstruction based on the dimensional characteristics of the spatial contour of the obstruction, and query the preset clamping force parameter table corresponding to the asset type to obtain the matching safe clamping force parameter as the flexible contact closure threshold.

[0045] The processor calculates the geometric dimensions of the obstruction based on the extracted spatial contour, including its length, width, and outline area. These dimensions are then matched with pre-stored dimensional templates for various types of metering assets to determine the asset type to which the obstruction belongs. Different types of metering assets, such as single-phase meters, three-phase meters, or current transformers, have different external dimensions, housing materials, and weight characteristics, thus requiring different safety clamping force parameters. If the clamping force is too small, the obstruction cannot be reliably clamped to perform the disturbance action, and the obstruction may slip during the disturbance; if the clamping force is too large, it may damage the injection-molded housing of the obstruction or cause damage to internal components.

[0046] After determining the asset type, a pre-stored clamping force parameter table is consulted. This parameter table was pre-established after clamping force calibration experiments on various types of measurable assets, and records the optimal clamping force range for each type of asset under the premise of reliable clamping without causing damage. The safe clamping force parameter matching the type of obstructing asset is obtained from this parameter table. And use it as the flexible contact closure threshold.

[0047] S3, control the electric gripper of the sorting robot to move to the physical coordinates of the grid, grip the obstruction with a flexible contact closure threshold, and perform planar disturbance action according to the horizontal micro-translation amount and fine-tuning rotation angle. At the same time, control the camera to perform high-frequency continuous capture of the area corresponding to the coordinates of the missing area to obtain a set of temporal images of the field of view with displacement process. In this embodiment, the processor generates movement control commands for the electric gripper based on the previously determined physical coordinates of the obstruction grid, controlling the electric gripper of the sorting robot to move from its current position to directly above the grid location where the obstruction is located. After the electric gripper reaches the target position, it closes the grid with a flexible contact threshold. As a force control target, the electric gripper is controlled to close its gripping fingers to grasp the obstruction. Among these, the flexible contact closure threshold is... As the force control setpoint of the gripper motor current loop, the gripper reliably holds the obstruction with this clamping force without slipping or damaging the housing.

[0048] Furthermore, according to the horizontal micro-distance translation (including and ) and fine-tuning the rotation angle The electric gripper is driven to perform a planar disturbance action that combines translation and rotation in the horizontal plane, causing the obstructing object to undergo a slight displacement within the grid safety boundary, thereby gradually exposing the target barcode area that was obscured.

[0049] It should be noted that during the planar disturbance process, the densely stacked injection-molded shell assets, after being frequently jolted by the high-speed reciprocating motion of the stacker crane, may experience deep interlocking of the reinforcing ribs, heat dissipation grooves, or anti-slip textures on the surfaces of adjacent assets, forming a static friction interlocking state. If the grippers ignore this jamming state and forcibly drag the asset, it can easily cause collateral topological damage. To solve this technical problem, such as... Figure 4 As shown, planar disturbance actions are performed according to the horizontal micro-translation and micro-adjustment rotation angle, including: S31, during the process of driving the electric gripper to move according to the horizontal micro-distance translation and fine-tuning rotation angle, the radial resistance feedback value of the electric gripper in the horizontal direction is collected in real time. The processor drives the electric gripper to perform a horizontal micro-translation. , and fine-tuning the rotation angle During the movement, the radial resistance feedback value of the electric gripper in the horizontal direction is collected in real time at a preset sampling frequency (e.g., 1000Hz). The radial resistance feedback value can be directly measured by a six-dimensional force sensor integrated into the end of the electric gripper, or indirectly estimated by combining the current loop feedback value of the servo motor with a dynamic model. It should be understood that the radial resistance feedback value characterizes the magnitude of the resistance experienced by the obstruction from adjacent assets during horizontal movement; a larger value indicates greater contact pressure and stronger friction between the obstruction and adjacent assets.

[0050] S32, when the radial resistance feedback value exceeds the preset static friction interlock threshold, it is determined that there is physical jamming between the obstruction and the adjacent asset, the electric gripper is controlled to stop the current movement, and the electric gripper is driven to output a high-frequency, low-amplitude escape oscillation wave in the horizontal direction to eliminate the surface static friction interlock state between the obstruction and the adjacent asset. Real-time collected radial resistance feedback value Interlock threshold with preset static friction Comparison. When At that time, it was determined that a physical jam had occurred between the obstruction and the adjacent asset. Among these, the static friction interlock threshold... This is a pre-calibrated critical resistance value, corresponding to the maximum allowable resistance under normal smooth sliding conditions. For example, it can be calibrated as the horizontal resistance value corresponding to 0.3 times the weight of the obstruction. When the radial resistance feedback value exceeds this threshold, it indicates that the surface static friction between the obstruction and the adjacent asset has exceeded the normal range, resulting in interlocking.

[0051] At this point, the control system pauses the current macroscopic movement of the electric gripper, which is based on the horizontal micro-translation and fine-tuning rotation angle. Instead, it drives the electric gripper to output a high-frequency, low-amplitude oscillation wave in the horizontal direction to help it overcome obstacles. It should be noted that the oscillation wave is a reciprocating micro-motion with a relatively high frequency (e.g., 20Hz to 50Hz) and a small amplitude (e.g., ±0.5mm to ±1mm), and its displacement function can be expressed as… ,in The range of fluctuation, The frequency is the oscillation frequency. High-frequency vibration generates minute alternating displacements at the contact interface between the obstruction and adjacent assets, causing continuous microscopic relative motion between the two contact surfaces that were originally in a static friction interlocking state. This transforms static friction into dynamic friction, disrupts the interlocking contact state between the anti-slip textures of the injection molded part, and eliminates the surface static friction interlocking.

[0052] S33, after detecting that the radial resistance feedback value has fallen back to the preset smooth sliding range, stop outputting the escape oscillation wave, and control the electric gripper to resume executing the remaining horizontal micro-translation and fine-tuning rotation angle until the planar disturbance action is completed.

[0053] The processor continuously monitors the radial resistance feedback value during the output of the escape oscillation wave. Changes. When monitored Return to the preset smooth sliding range (in This is the upper limit threshold for the smooth sliding range, and its value is less than the static friction interlock threshold. When the resistance value is within 0.1 times the weight of the obstruction (e.g., the horizontal resistance value can be calibrated to be within the range of the horizontal resistance value corresponding to 0.1 times the weight of the obstruction), it is determined that the obstruction has been freed from the stuck state. The output of the freeing shock wave is stopped, and the electric gripper is controlled to start from the current pause position to resume the execution of the remaining unfinished horizontal micro-translation and fine-tuning rotation angle until all disturbance actions are completed.

[0054] Through the aforementioned force control sensing and escape oscillation processing, the interlocking state can be automatically released when physical jamming occurs, avoiding the associated topological damage caused by forced dragging.

[0055] While performing the aforementioned perturbation actions, the processor simultaneously controls the global overhead camera to continuously capture images of the spatial region corresponding to the coordinates of the incomplete area at a preset high frame rate (e.g., more than 30 frames per second). This is to completely record the spatial position changes of the occluder at each key position during the perturbation displacement process, thereby obtaining a set of temporal images of the field of view showing the displacement process of the occluder. ,in This is the first frame image before the disturbance begins. This is the last frame image when the disturbance action is completed.

[0056] In addition, to ensure the temporal consistency of subsequent image stitching, the camera's capture trigger signal and the electric gripper's motion control command are synchronized in time. Each capture is triggered after the electric gripper moves a predetermined microstep, ensuring that each frame in the above-mentioned temporal image set can completely correspond to the intermediate position state during the displacement process of the occluder.

[0057] S4. Using the fixed physical boundary as the spatial alignment anchor point, extract the newly exposed local feature slices in each frame of the visual temporal image set due to the displacement of the occluded object, and perform pixel-level topological recombination of all local feature slices to generate a complete barcode reconstruction image, which is then input into the decoding engine for parsing.

[0058] In this embodiment, because the obstruction undergoes spatial displacement during the disturbance, and the camera may experience slight jitter or positional shifts during each frame capture, the images in each frame are not naturally aligned. Simply superimposing the images directly would result in blurred or distorted reconstructed images due to spatial misalignment. Furthermore, the displacement of the obstruction is continuous and gradual; each frame only exposes a partial fragment of the barcode. These partial fragments need to be extracted from multiple frames and stitched together to form a complete barcode image. Therefore, the processor uses the aforementioned fixed physical boundary as a reference anchor point for spatial alignment. It should be understood that this boundary is a rigid edge of the asset containing the target barcode, remaining stationary throughout the disturbance process and not changing with the displacement of the obstruction. Therefore, it can serve as a stable reference for spatial registration between the images in each frame.

[0059] To accurately extract newly exposed local feature slices from the temporal image set of the visual field, optionally, the fixed physical boundary is used as a spatial alignment anchor point to extract newly exposed local feature slices from the temporal image set of the visual field, including: S41, take the first frame image in the visual temporal image set as the reference frame, extract the feature points of the fixed physical boundary in each subsequent frame image and perform feature matching with the reference frame, and calculate the spatial alignment transformation matrix of each frame image relative to the reference frame. The processor will collect the temporal images of the field of view. The first frame image As a baseline frame, the occluding object has not yet shifted, and the unoccluded portion of the target barcode remains in its original state. For subsequent frames... ( ), extract the feature points of the fixed physical boundaries respectively, and compare them with the reference frame. The feature points corresponding to the fixed physical boundary are matched.

[0060] Optionally, feature point extraction and matching can employ scale-invariant feature transformation algorithms or accelerated robust feature algorithms. These algorithms can detect significant features such as corner points and edge points in the image and generate feature descriptors with rotation and scale invariance. Cross-frame matching is achieved by calculating the Euclidean distance or Hamming distance between feature descriptors.

[0061] After feature point matching is completed, the spatial alignment transformation matrix of each frame image relative to the reference frame is calculated based on the correspondence between the matched point pairs. The spatial alignment transformation matrix is ​​typically an affine transformation matrix or a perspective transformation matrix, and its form can be expressed as:

[0062] in, Describe rotation and scaling. Describe translation, Describes perspective distortion. This matrix describes the differences in translation, rotation, scaling, and perspective relationships between each frame of an image relative to a reference frame, and is used to register the images of each frame to a unified coordinate system.

[0063] S42, after each frame image is registered by the spatial alignment transformation matrix, pixel difference calculation is performed between each frame image and the reference frame in sequence to identify pixel clusters located within the coordinate range of the incomplete region and significantly different due to the displacement of the occluder, as newly exposed local feature slices, and the spatial positioning coordinates corresponding to each local feature slice are recorded.

[0064] The processor uses the spatial alignment transformation matrix obtained from the aforementioned calculation. For subsequent frames of images Perform coordinate transformations separately and register them to the reference frame. In the coordinate system, the registered image is obtained. Then register each frame of the image. sequentially with the reference frame Perform pixel difference calculation:

[0065] in, For pixel coordinates, This is the difference result. Due to the reference frame... The record shows the initial state before the occlusion is displaced, while subsequent registration frames... It records the state during the displacement of the occluder. The pixel difference between the two mainly comes from the barcode area that is newly exposed due to the movement of the occluder. That is, the pixel area that was originally covered by the occluder, was not visible in the reference frame, but became visible in subsequent frames.

[0066] Within the coordinate range of the incomplete region, identify the difference results. Clusters of consecutive pixels whose pixel values ​​exceed a preset difference threshold are considered as newly exposed local feature slices. And record the spatial positioning coordinates corresponding to each local feature slice. The spatial positioning coordinates can be obtained by combining the pixel position of the slice in the registered image with the pixel coordinate-to-world coordinate mapping established in step S13. These coordinates are used in subsequent step S44 to determine the fill position of the slice on the reconstructed canvas. It should be noted that as the occluder gradually moves, the local feature slices extracted from different frames correspond to different local regions of the barcode, collectively covering various parts of the incomplete area.

[0067] After extracting the local feature slices, the processor performs pixel-level topological reconstruction to generate a complete barcode reconstruction image. Optionally, pixel-level topological reconstruction is performed on all local feature slices to generate a complete barcode reconstruction image, including: S43, establish a blank reconstruction canvas, and map the original pixel area of ​​the target barcode in the reference frame that is not occluded to the corresponding position on the reconstruction canvas; The processor creates a blank reconstruction canvas. Its size is consistent with the complete theoretical area size of the target barcode. The reference frame... The original pixel region of the target barcode that is not occluded (i.e., the non-incomplete region from which valid barcode features can be extracted in step S13) is mapped to the reconstruction canvas according to its original pixel position. The corresponding positions. It should be understood that these original pixel areas are the parts of the barcode that are never obscured, forming the static basis of the reconstructed image.

[0068] S44, according to the spatial positioning coordinates, all the local feature slices are sequentially filled into the empty areas in the reconstruction canvas. When there is edge overlap between adjacent local feature slices, a pixel weighted fusion algorithm is used to eliminate the splicing seams, and finally the complete barcode reconstruction image with the occlusion removed is output.

[0069] The processor slices the data based on the recorded local features. Corresponding spatial positioning coordinates Fill all local feature slices sequentially into the reconstruction canvas. The missing area within. During this process, because the displacement of the occluder is continuous, the newly exposed area between adjacent frames may have a small amount of edge overlap, that is, barcode fragments at the same physical location may be partially captured in two adjacent frames. When edge overlap is encountered between adjacent local feature slices, a pixel-weighted fusion algorithm is used to eliminate the stitching seams. Specifically, for any pixel position within the overlapping area... The merged pixel values ​​are calculated using the following formula:

[0070] in, The number of local feature slices covering this pixel location. For the first The pixel value of a slice at that pixel location. This refers to the fusion weight assigned to the pixel location within the slice. The weight coefficients are distributed based on the pixel's distance from the slice's boundary; specifically, pixels closer to the slice center have higher weights, and pixels closer to the slice boundary have lower weights. For example, for a local feature slice... Its weighting function It can be defined as a monotonically increasing function of the distance from the pixel to the slice boundary, so that pixels closer to the inside of the slice in the overlapping region have a higher contribution and pixels closer to the edge of the slice have a lower contribution, thereby achieving a smooth transition in the overlapping region and avoiding obvious splicing marks.

[0071] After the pixel-level topological reconstruction described above, a complete barcode reconstruction image with occlusions removed is output. In this image, all areas of the target barcode (including the original unobstructed portion and the incomplete areas reconstructed through multi-frame stitching) are fully rendered. The processor inputs the reconstructed image into the decoding engine for barcode parsing. The decoding engine detects the start and end characters of the barcode and the bar-space width sequence, converting the image information into corresponding numerical or character information, thereby obtaining the identity information carried by the target barcode.

[0072] This invention also provides an electronic device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions. When the computer program instructions are executed by the processor, they perform the operations that the processor is configured to perform as described in any of the preceding claims.

[0073] This invention also provides a computer-readable storage medium storing an executable computer program whose instructions, when executed by a processor, implement the operations that the processor is configured to perform as described in any of the preceding claims.

[0074] This invention also provides a computer program product comprising an executable computer program whose instructions, when executed by a processor, implement the operations that the processor is configured to perform as described in any of the preceding claims.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An outdoor prefabricated automated metering warehouse, characterized in that, Includes a processor, a camera, and a sorting robot, wherein the processor is configured to perform the following operations: S1, control the camera to acquire a global overhead image of the turnover box and perform target detection. When it is determined that there is a target barcode that fails to be decoded due to physical obstruction, extract the coordinates of the missing area of ​​the target barcode and the fixed physical boundary that is not obstructed, and calculate the spatial contour of the obstruction and its grid physical coordinates in the turnover box. S2, based on the spatial contour of the obstruction, the physical coordinates of the grid, and the preset grid boundary margin, calculate the micro-intrusion disturbance vector parameters for the obstruction, including the flexible contact closure threshold, the horizontal micro-translation amount, and the fine-tuning rotation angle, and the spatial motion envelope formed by the horizontal micro-translation amount and the fine-tuning rotation angle is constrained within the grid boundary margin; S3, control the electric gripper of the sorting robot to move to the physical coordinates of the grid, grip the obstruction with a flexible contact closure threshold, and perform planar disturbance action according to the horizontal micro-translation amount and fine-tuning rotation angle. At the same time, control the camera to perform high-frequency continuous capture of the area corresponding to the coordinates of the missing area to obtain a set of temporal images of the field of view with displacement process. S4. Using the fixed physical boundary as the spatial alignment anchor point, extract the newly exposed local feature slices in each frame of the visual temporal image set due to the displacement of the occluded object, and perform pixel-level topological recombination of all local feature slices to generate a complete barcode reconstruction image, which is then input into the decoding engine for parsing.

2. The outdoor prefabricated automated metering warehouse according to claim 1, characterized in that, Determining the existence of target barcodes that fail to decode due to physical obstruction includes: S11, extract the preset barcode attention area corresponding to each measurement asset in the global overhead image and decode it. When the decoding of any preset barcode attention area fails, extract the edge contour features of the preset barcode attention area and its surrounding area. S12, based on the edge contour features, determine whether there is a physical outer contour of an adjacent asset that intrudes into the boundary of the preset barcode interest area. If so, determine that the barcode in the preset barcode interest area is a target barcode that fails to be decoded due to physical occlusion.

3. The outdoor prefabricated automated metering warehouse according to claim 2, characterized in that, Extract the coordinates of the missing area and the fixed physical boundary of the unoccluded area of ​​the target barcode, and calculate the spatial contour of the obstruction causing the obstruction and its physical coordinates within the grid of the turnover box, including: S13, using at least one unobstructed edge of the physical outer contour of the asset containing the target barcode as a fixed physical boundary, extracting the pixel coordinates of the fixed physical boundary in the global overhead image, and mapping it to the world coordinate system in combination with the camera's intrinsic parameters and hand-eye calibration matrix; determining the complete theoretical region of the target barcode based on the relative position of the fixed physical boundary and the preset barcode interest area, dividing the local area within the complete theoretical region that is obstructed and cannot extract effective barcode features into incomplete regions and extracting their pixel coordinates; S14, using the pixel coordinates of the incomplete area as an index, detect the physical outer contour edge of the adjacent asset that intrudes into the area, extract the contour line segment located in the area as the spatial contour of the occlusion, and determine its grid physical coordinates in the turnover box based on the centroid position of the spatial contour of the occlusion.

4. The outdoor prefabricated automated metering warehouse according to claim 1, characterized in that, Based on the spatial contour of the occlusion, the physical coordinates of the mesh, and the preset mesh boundary margin, calculate the micro-intrusion perturbation vector parameters for the occlusion, including: S21, calculate the theoretical amount of motion required to expose the target barcode based on the coordinates of the missing area, and use the spatial contour of the obstruction and the preset grid boundary margin to determine the maximum safe activity range under the grid physical coordinates, and limit and truncate the theoretical amount of motion to output the horizontal micro-translation amount and fine-tuning rotation angle. S22, determine the asset type of the obstruction based on the dimensional characteristics of the spatial contour of the obstruction, and query the preset clamping force parameter table corresponding to the asset type to obtain the matching safe clamping force parameter as the flexible contact closure threshold.

5. The outdoor prefabricated automated metering warehouse according to claim 1, characterized in that, Perform planar disturbance actions according to the horizontal micro-translation amount and fine-tuning rotation angle, including: S31, during the process of driving the electric gripper to move according to the horizontal micro-distance translation and fine-tuning rotation angle, the radial resistance feedback value of the electric gripper in the horizontal direction is collected in real time. S32, when the radial resistance feedback value exceeds the preset static friction interlock threshold, it is determined that there is physical jamming between the obstruction and the adjacent asset, the electric gripper is controlled to stop the current movement, and the electric gripper is driven to output a high-frequency, low-amplitude escape oscillation wave in the horizontal direction to eliminate the surface static friction interlock state between the obstruction and the adjacent asset. S33, after detecting that the radial resistance feedback value has fallen back to the preset smooth sliding range, stop outputting the escape oscillation wave, and control the electric gripper to resume executing the remaining horizontal micro-translation and fine-tuning rotation angle until the planar disturbance action is completed.

6. The outdoor prefabricated automated metering warehouse according to claim 1, characterized in that, Using the fixed physical boundary as the spatial alignment anchor point, newly exposed local feature slices are extracted from each frame of the temporal image set of the field of view, including: S41, take the first frame image in the visual temporal image set as the reference frame, extract the feature points of the fixed physical boundary in each subsequent frame image and perform feature matching with the reference frame, and calculate the spatial alignment transformation matrix of each frame image relative to the reference frame. S42, after each frame image is registered by the spatial alignment transformation matrix, pixel difference calculation is performed between each frame image and the reference frame in sequence to identify pixel clusters located within the coordinate range of the incomplete region and significantly different due to the displacement of the occluder, as newly exposed local feature slices, and the spatial positioning coordinates corresponding to each local feature slice are recorded.

7. An outdoor prefabricated automated metering warehouse according to claim 6, characterized in that, Pixel-level topological reconstruction of all local feature slices is performed to generate a complete barcode reconstruction image, including: S43, establish a blank reconstruction canvas, and map the original pixel area of ​​the target barcode in the reference frame that is not occluded to the corresponding position on the reconstruction canvas; S44, according to the spatial positioning coordinates, all the local feature slices are sequentially filled into the empty areas in the reconstruction canvas. When there is edge overlap between adjacent local feature slices, a pixel weighted fusion algorithm is used to eliminate the splicing seams, and finally the complete barcode reconstruction image with the occlusion removed is output.

8. An electronic device, characterized in that, The electronic device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, which, when executed by the processor, implement the operation that the processor is configured to perform as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an executable computer program that, when executed by a processor, performs the operation that the processor is configured to perform as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes an executable computer program that, when executed by a processor, implements the operations that the processor is configured to perform as claimed in any one of claims 1 to 7.

Images