First aid material image recognition and inventory method and system based on deep learning

Abstract

This application provides a method and system for image recognition and inventory of emergency supplies based on deep learning, relating to the fields of image recognition and supplies management technology. This application simultaneously acquires surface texture images and spatial depth images of the storage area through a sensing device, establishing a three-dimensional coordinate system for the storage space. The two types of images are input into a deep learning model to identify the boundary lines caused by mutual occlusion of supplies under a spatial position reference, and the visible edge contours of each individual supply are located by combining depth numerical jumps. The visible edge contours are fitted with a standard specification model to reconstruct the complete occupied volume. The stacking level is determined based on the physical occupancy order of the complete occupied volume in the vertical direction, thereby counting the actual number of covered supplies in the warehouse. Based on the actual number in the warehouse and consumption prediction data, target supplies in a state of supply imbalance are identified and a supply replenishment signal is generated, which can improve the accuracy and real-time performance of emergency supplies inventory.

Description

Technical Field

[0001] This application relates to the fields of image recognition and material management technology, and in particular to a method and system for image recognition and inventory of emergency supplies based on deep learning. Background Technology

[0002] Emergency medical supplies are critical resources for responding to emergency rescue events, and the timeliness of their storage, inventory, and supply directly impacts the efficiency of emergency response missions. Currently, the inventory and management of emergency medical supplies mainly rely on manual counting or registration using methods such as 2D barcodes and RFID. However, emergency medical supplies in storage areas often exhibit characteristics such as multiple categories coexisting, stacking, and varying shapes, making manual counting inefficient, error-prone, and difficult to update inventory status in real time.

[0003] To address these issues, some existing solutions employ single-vision image recognition technology for material identification and inventory, extracting features from two-dimensional images to identify material types and locations. However, this approach struggles to accurately determine the actual presence of underlying or obscured materials in scenarios where they are stacked or obscured, leading to missed detections and inaccurate inventory counts. Furthermore, most of these solutions perform only static identification and counting, lacking dynamic correlation analysis with emergency response mission consumption predictions. This prevents timely responses to dynamic discrepancies between material inventory levels and demand changes, resulting in insufficient decision support for emergency material replenishment.

[0004] Therefore, how to solve the problems existing in the current solutions, improve the identification accuracy and inventory completeness of emergency supplies in stacked storage scenarios, and achieve dynamic matching between the quantity in stock and consumption demand, thereby improving the timeliness and accuracy of emergency supplies supply decisions, is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a deep learning-based method and system for image recognition and inventory of emergency supplies, which can solve the problems existing in the existing related solutions, significantly improve the ability to identify and replenish occluded emergency supplies in stacked storage scenarios, and achieve dynamic matching between the actual inventory quantity and the predicted consumption demand, thereby improving the accuracy of emergency supply inventory and the timeliness of replenishment decisions. The specific solution is as follows: Firstly, this application provides a deep learning-based method for image recognition and inventory of emergency supplies, including: By using sensing devices to simultaneously acquire surface texture images and spatial depth images of the storage area, a three-dimensional coordinate system for the material storage space is established. This three-dimensional coordinate system is used to establish the spatial position reference for individual materials. The surface texture image and spatial depth image are input into a deep learning model to identify the boundary edge lines caused by mutual occlusion of materials under the spatial position reference, and the visible edge contours of each material individual are located by combining the depth value jump. The visible edge contour is fitted to a standard specification model using the deep learning model to derive the spatial occupancy completion parameters of the occupant. These spatial occupancy completion parameters are used to restore the complete volume occupied by the material. Based on the physical shielding order of the complete occupied volume in the vertical direction, the stacking level of the material is determined, and the stacking level is used to count the actual number of covered materials in the warehouse. The actual inventory quantity is spatiotemporally correlated with the consumption prediction data of related emergency rescue missions. The dynamic matching deviation between the evolution trend of the inventory quantity and the consumption prediction demand is analyzed to identify target materials in a state of supply imbalance. Extract the missing information of the target materials to generate a material replenishment signal and send it to the management terminal for synchronous updating of the digital material dashboard.

[0006] Optionally, the step of identifying the boundary edge line caused by mutual occlusion of materials under the spatial location reference, and locating the visible edge contour of each individual material in combination with depth numerical jumps, includes: Identify the locations where the continuity of pixel attributes is interrupted in the surface texture image to extract preliminary boundary edges; The preliminary boundary edge is mapped onto the spatial depth image, and planar interference textures are removed based on the abrupt change in depth values ​​on both sides of the preliminary boundary edge to establish the boundary edge line describing the physical boundary of the material individual. Using the boundary line as the boundary, a closed region with depth values ​​in the same continuous interval is extracted under the spatial location reference, which serves as the visible edge contour of each individual material.

[0007] Optionally, fitting the visible edge contour to a standard specification model using the deep learning model includes: Based on the geometric shape of the visible edge contour of each individual material, the target standard model corresponding to the geometric shape is retrieved from the pre-stored material database. Adjust the rotation angle and scaling ratio of the target standard model in the three-dimensional coordinate system until the top projection shape of the target standard model under the spatial position reference achieves shape coverage matching with the visible edge contour.

[0008] Optionally, the derivation of spatial occupancy completion parameters for the obscured portion, wherein the spatial occupancy completion parameters are used to restore the complete volume occupied by the individual material, include: Starting from the top surface of the target standard model after shape coverage matching, probe downwards along the vertical axis of the three-dimensional coordinate system to the bottom support plane or the top surface of the lower material. Calculate the extension displacement along the vertical axis and use the extension displacement as the spatial occupancy completion parameter for the obscured part of the material individual; Based on the bottom shape of the target standard model and the space occupancy completion parameters, the physical form of the occupied part is derived and combined with the visible edge contour to form the complete occupied volume.

[0009] Optionally, determining the stacking level of individual materials based on the physical shielding order of the complete occupied volume in the vertical direction includes: Calculate the overlapping projected area of ​​the complete occupied volume of each material individual on the horizontal plane of the three-dimensional coordinate system; For individual materials whose overlapping projection area exceeds a preset range, the volumetric center of gravity height of each individual material is extracted; A vertical shielding sequence is established according to the volume center height from largest to smallest. The vertical shielding sequence is used to determine the stacking level to which each material belongs.

[0010] Optionally, the actual quantity of the covered materials in stock includes: Identify the bottom-level material individuals that are not in the first-order level of the stacking hierarchy; Analyze the viewpoint exposure ratio of the underlying material individuals in the surface texture image; If the viewpoint exposure ratio is lower than the preset display extreme value, the physical existence of the bottom layer material individual is determined based on the vertical occlusion sequence, and the total number of materials recorded in all stacking levels is counted to obtain the actual quantity in stock.

[0011] Optionally, the step of spatiotemporally correlating the actual inventory quantity with the consumption prediction data of related emergency rescue tasks includes: Obtain the historical consumption frequency of related emergency rescue tasks, and map the historical consumption frequency to a preset future task cycle; Based on the urgency of the current emergency rescue mission, the consumption growth rate of each type of material is determined within the future mission cycle. This consumption growth rate is used to generate consumption forecast data describing the demand at different time points.

[0012] Optionally, the analysis of the dynamic matching deviation between the evolution trend of inventory quantity and the predicted consumption demand identifies target materials in a state of supply imbalance, including: Based on the actual inventory quantity, establish a trend line for the decrease in inventory over time, and establish a corresponding trend line for the increase in demand based on the consumption forecast data. Calculate the spatiotemporal deviation between the residual reduction trend line and the demand growth trend line; If the degree of spatiotemporal deviation indicates that the remaining quantity reduction trend line drops below the preset inventory threshold within the preset replenishment cycle, then the corresponding material is determined to be in a state of supply imbalance.

[0013] Optionally, before generating a material replenishment signal by extracting the shortage information of the target material, the method further includes: Obtain real-time task attribute information for the associated emergency medical area; The shortage information of the target materials is combined with the real-time task attribute information to perform demand coupling analysis. The demand coupling analysis is used to determine the urgency level of each target material for the current task. Based on the supply urgency level, the hierarchical response order of each target material in the resource allocation and circulation is determined, and the hierarchical response order is used as the decision guidance parameter for the material supply signal.

[0014] Secondly, this application provides a deep learning-based image recognition and inventory system for emergency medical supplies, including: A module is established to synchronously acquire surface texture images and spatial depth images of the storage area using sensing devices, and to establish a three-dimensional coordinate system for the material storage space. The three-dimensional coordinate system is used to establish the spatial position reference of individual materials. The recognition module is used to input the surface texture image and spatial depth image into a deep learning model, identify the boundary edge line generated by mutual occlusion of materials under the spatial position reference, and locate the visible edge contour of each material individual by combining the depth value jump. The derivation module is used to fit the visible edge contour with the standard specification model using the deep learning model, and derive the spatial occupancy completion parameters of the occupant. The spatial occupancy completion parameters are used to restore the complete volume occupied by the material. The statistics module is used to determine the stacking level of individual materials based on the physical obscuring order of the complete occupied volume in the vertical direction. The stacking level is used to count the actual number of covered materials in the warehouse. The analysis module is used to spatially and temporally correlate the actual inventory quantity with the consumption prediction data of related emergency rescue missions, analyze the dynamic matching deviation between the evolution trend of inventory quantity and the consumption prediction demand, and identify target materials in a state of supply imbalance. The generation module is used to extract the missing information of the target materials, generate a material replenishment signal, and send it to the management terminal for synchronously updating the digital material dashboard.

[0015] The beneficial effects of this application are: This application acquires surface texture images and spatial depth images simultaneously using sensing devices, and establishes a three-dimensional coordinate system as the spatial position benchmark for individual materials, thus providing a unified reference system for subsequent edge recognition and contour localization. Then, a deep learning model is used to identify the boundary edge line under this spatial position benchmark and combine it with depth jumps to locate the visible edge contour. Next, the complete occupied volume of individual materials is restored by fitting a standard specification model and deriving spatial occupancy completion parameters. After that, the stacking level is determined based on the vertical occupancy order of the complete occupied volume, and the actual number of covered materials in the warehouse is counted. Finally, the actual number in the warehouse is spatiotemporally correlated with consumption prediction data to identify target materials with supply imbalances and generate material replenishment signals.

[0016] This approach addresses the issues present in existing solutions, significantly enhances the ability to identify and replenish obscured emergency supplies in stacked storage scenarios, and enables dynamic matching between actual stock levels and predicted consumption needs. This, in turn, improves the accuracy of emergency supplies inventory and the timeliness of replenishment decisions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a deep learning-based method for image recognition and inventory of emergency supplies, provided as an embodiment of this application; Figure 2 A schematic diagram illustrating the establishment of a three-dimensional coordinate system and spatial location reference for a storage area, provided in an embodiment of this application; Figure 3 A flowchart illustrating a visible edge contour localization process provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the fitting of a standard specification model and the restoration of the complete occupied volume, provided for an embodiment of this application; Figure 5 A schematic diagram illustrating a process for determining stacking levels and counting the number of covered materials, provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the matching relationship between a residual reduction trend line and a demand growth trend line, provided for an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a deep learning-based emergency medical supplies image recognition and inventory system provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In existing solutions, manual counting and inventory methods based on two-dimensional barcodes and radio frequency identification are inefficient and prone to errors. In contrast, single visual recognition solutions are difficult to accurately obtain the actual presence of underlying or obscured materials when faced with stacked or obscured materials, which can easily lead to missed detections and inaccurate inventory counts. Furthermore, these solutions lack dynamic correlation analysis with emergency mission consumption predictions and cannot respond promptly to dynamic deviations between material inventory and demand changes.

[0021] To address this, this application provides a deep learning-based image recognition and inventory scheme for emergency medical supplies, which can solve the problems existing in the current related schemes, significantly improve the ability to identify and complete emergency medical supplies that are obscured in stacked storage scenarios, and realize dynamic matching between the actual quantity in stock and the predicted consumption demand, thereby improving the accuracy of emergency medical supplies inventory and the timeliness of replenishment decisions.

[0022] See Figure 1 As shown in the figure, this application discloses a method for image recognition and inventory of emergency supplies based on deep learning. Figure 1 The flowchart presents the six steps in the order of S11 to S16: image acquisition and establishment of a 3D coordinate system, identification of boundary lines and visible edge contours, fitting of standard specification models and derivation of complete occupied volume, determination of stacking levels and statistics of actual inventory quantity, spatiotemporal correlation analysis to identify target materials, and generation of material replenishment signals and updating of the dashboard. This visually reflects the execution sequence of each step in the method. Specific steps include: Step S11: Simultaneously acquire surface texture images and spatial depth images of the storage area using sensing devices, and establish a three-dimensional coordinate system for the material storage space. The three-dimensional coordinate system is used to establish the spatial position reference of individual materials.

[0023] In this embodiment, a sensing device is first used to synchronously collect data on the storage area of ​​emergency supplies to obtain a surface texture image that reflects the color and texture information of the supplies, and a spatial depth image that reflects the distance distribution between the supplies and the sensing device. It should be understood that the surface texture image refers to a two-dimensional image that reflects the appearance characteristics of the supplies, acquired through a color imaging channel; the spatial depth image refers to an image acquired through a depth imaging channel where each pixel carries a distance value. The sensing device may include a color camera and a depth sensor, both of which are synchronously triggered to ensure spatial and temporal alignment of the two types of images.

[0024] See Figure 2 As shown, Figure 2 The spatial relationship between the sensing device, the surface texture image acquisition channel, the spatial depth image acquisition channel, the storage area, and the three-dimensional coordinate system established on the storage area is demonstrated. The origin O of the coordinate system is a corner of the storage area, with the two horizontal axes being the X-axis 21 and Y-axis 22, and the vertical axis being the Z-axis 23. The individual material 24 is located within the coordinate system to represent its spatial position reference.

[0025] After image acquisition, a three-dimensional coordinate system is established, using a fixed reference point within the storage area as the origin, the length and width of the storage area as the horizontal axes, and the opposite direction of gravity as the vertical axis. This system covers the entire material storage space. Using this system, pixel positions in the surface texture image and depth values ​​in the spatial depth image can be mapped to a unified three-dimensional coordinate system, thus establishing the spatial position reference of individual materials within the storage space. This spatial position reference provides a unified reference for subsequent steps such as edge recognition, contour localization, standard model fitting, and stacking level determination, avoiding matching deviations caused by processing data under different coordinate systems.

[0026] Because the materials are stacked and obstructed within the storage space, the spatial location alone is insufficient to separate individual materials from the image; therefore, it is necessary to further identify the physical boundaries between the materials.

[0027] Step S12: Input the surface texture image and spatial depth image into the deep learning model, identify the boundary edge line caused by mutual occlusion of materials under the spatial position reference, and locate the visible edge contour of each material individual by combining the depth value jump.

[0028] In this embodiment, after establishing a three-dimensional coordinate system, since emergency supplies are often stacked or juxtaposed in the storage area, relying solely on two-dimensional texture information can easily misjudge the printed texture or light and shadow changes on the surface of the supplies as the edge of the object. Therefore, it is necessary to input the surface texture image and the spatial depth image into the deep learning model simultaneously, and use the depth information to help identify the real physical boundary.

[0029] It should be noted that the deep learning model here uses an encoder-based approach. The semantic segmentation network with a decoder structure, specifically using U The segmentation network, improved based on the Net structure, consists of multiple convolutional layers, downsampling layers, and upsampling layers. The input is a multi-channel tensor obtained by concatenating surface texture images and spatial depth images, and the output is a pixel-wise edge probability map and a material region segmentation map. Before deployment, the model is trained in a supervised manner using a sample set with edge and material region annotations. During training, the cross-entropy loss function is used to update the network weights by minimizing the loss function, thereby enabling the model to have the joint discrimination ability of texture information and depth information. The specific network depth and number of channels can be set according to the actual scenario.

[0030] See Figure 3 As shown, Figure 3 The process flow from surface texture image and spatial depth image to visible edge contour is illustrated in the form of a flowchart, which sequentially presents four processing nodes: pixel attribute continuity interruption identification 31, depth value abrupt change magnitude judgment 32, boundary edge line establishment 33, and closed region extraction 34. Specifically, this step may include the following sub-steps: First, identify the locations where the continuity of pixel attributes is interrupted in the surface texture image to extract preliminary boundary edges. That is... Figure 3 The processing corresponding to node 31. Specifically, the deep learning model performs pixel-by-pixel analysis on the pixel attributes of the surface texture image, such as grayscale, color channel values, and gradients. When adjacent pixels show significant discontinuities in color or gradient, this location is used as a candidate point for the initial boundary. To quantify the degree of discontinuity in pixel attribute continuity, a gradient magnitude formula can be used for discrimination, the expression of which is as follows: ; In the formula, Represents pixels gradient magnitude at that point and These represent the gradient components of the pixel in the horizontal and vertical directions, respectively, obtained by convolving the surface texture image using the Sobel operator; when When the gradient value exceeds a preset gradient threshold, it is determined that there is a break in the continuity of pixel attributes at that pixel point, and it is included in the preliminary edge boundary. The gradient calculation described above is a common image edge detection formula, and will not be elaborated further here. The preset gradient threshold can be set according to the noise level of the surface texture image, for example, as a several times the average gradient magnitude of the image.

[0031] Secondly, the preliminary edges obtained solely from texture information may contain a large amount of planar interference texture caused by surface printing or shadows; therefore, depth information is needed for filtering. Figure 3 The processing corresponding to node 32 involves mapping the initial boundary edge onto the spatial depth image, removing planar interference textures based on the abrupt changes in depth values ​​on both sides of the initial boundary edge, and establishing the boundary edge line describing the physical boundary of the individual material. Figure 3 Node 33.

[0032] Specifically, for each initial boundary edge, depth values ​​are sampled along the normal direction on both sides and the difference is calculated. If the difference exceeds a preset threshold, it is determined that there is a real physical height difference at that location, and it is retained as a boundary edge line; otherwise, it is considered planar interference texture and discarded. The calculation of the depth value abrupt change amplitude can be performed using the following expression: ; In the formula, This represents the magnitude of the depth abrupt change at pixel p on the initial boundary edge. and These represent the depth values ​​sampled from both sides of pixel p along the edge normal; when When the threshold is exceeded, it is retained as the boundary line. This threshold can be set as a fraction of the average thickness of the materials in the storage space, and can be specifically set according to the characteristics of the material category.

[0033] Third, using the boundary line as the boundary, closed regions with depth values ​​within the same continuous interval are extracted under the spatial location benchmark, serving as the visible edge contours of each individual material. Figure 3 Node 34.

[0034] Specifically, using boundary lines as constraints for region segmentation, region growing is performed along the depth values ​​in the spatial depth image: pixels with depth value differences within a preset continuous range are grouped into the same continuous region, and enclosed by the corresponding boundary lines to form closed regions; each closed region obtained is the visible edge contour of the unobstructed portion of an individual object from the current viewpoint. The criteria for region growing are: ; In the formula, Seed pixels, for The neighboring pixels, and These represent the depth values ​​of the two pixels, This is a preset depth continuity threshold; when this inequality is satisfied, Inclusion and If the same continuous region is not reached, growth will stop.

[0035] By combining the two-dimensional texture edge information with the three-dimensional depth transition information through the above steps, the physical occlusion boundaries between materials can be effectively separated from the complex texture background, providing accurate contour input for subsequent standard model fitting. Since the visible edge contours can only reflect the visible part of an individual material, and the shape of the occluded part in a stacked scene cannot be directly obtained from the image, the next step is to restore the complete shape of the materials through standard specification model fitting.

[0036] Step S13: Use a deep learning model to fit the visible edge contour with the standard specification model to derive the space occupancy completion parameters of the occupancy part. The space occupancy completion parameters are used to restore the complete volume occupied by the material.

[0037] In this embodiment, since only the visible edge outline of individual supplies can usually be obtained in a stacked scenario, the shape of the occluded parts cannot be directly obtained from the image data. Therefore, it is necessary to use pre-stored standard specification models to complete the actual shape of the supplies. The standard specification model refers to a set of three-dimensional models pre-established for the geometric specifications of different types of emergency supplies. Each type of supply corresponds to one or more standard models, which are used to describe the complete shape of the supply when it has not been deformed. For example, the standard length, width, height and bottom shape of emergency supplies such as medicine boxes, bandage boxes, and infusion bags.

[0038] See Figure 4 As shown, Figure 4 The derivation relationships between visible edge contour 41, material library 42, target standard model 43, shape coverage matching 44, spatial occupancy completion parameters 45, and complete occupied volume 46 are illustrated in schematic form. Specifically, this step may include the following sub-steps: First, based on the geometric shape of the visible edge contour 41 of each individual material, a target standard model 43 corresponding to the geometric shape is retrieved from the pre-stored material database 42. Specifically, a deep learning model extracts the geometric features of the visible edge contour, such as the aspect ratio, number of sides, and curvature distribution of the outer contour, and performs similarity matching with the top projection features of each standard model in the material database. The similarity can be calculated using the well-known cosine similarity formula, the expression of which is: ; In the formula, and Let A and B be the geometric feature vectors of the visible edge contour and the geometric feature vector of the projection of the top surface of the standard model, respectively, and let A·B represent the dot product of the two vectors. and Let these represent the magnitudes of the two vectors, The closer the value is to 1, the more similar the two are. The standard model with the highest similarity (sim) is selected as the target standard model 43. The geometric specifications of various materials pre-stored in the material warehouse can be pre-entered according to the product parameters provided by the supplier in actual applications. Specifically, they can be set according to the actual categories of emergency supplies.

[0039] Then, shape coverage matching 44 is performed, that is, adjusting the rotation angle and scaling ratio of the target standard model 43 in the three-dimensional coordinate system until the top projection shape of the target standard model 43 under the spatial position reference achieves shape coverage matching with the visible edge contour 41. Specifically, using the geometric center of the visible edge contour as the fitting reference point, a rotation transformation about the vertical axis and a scaling transformation along the horizontal direction are applied to the target standard model. The coordinate transformation expression of any point on the top projection of the target standard model after rotation and scaling is: ; In the formula, The coordinates of the original point on the top surface projection of the target standard model. These are the coordinates of the corresponding points after the transformation. For scaling ratio, Let be a two-dimensional rotation matrix about the vertical axis. Let be the rotation angle of the target standard model about the vertical axis. This is the translation vector. The overlap between the model's top surface projection and the visible edge contour is iteratively compared. When the overlap reaches the shape coverage matching condition (i.e., the deviation between the projected contour and the visible edge contour is below a preset tolerance range), the current rotation angle and scaling ratio are fixed as the basis for deriving subsequent spatial occupancy completion parameters. The overlap can be measured using the intersection-union ratio (IUU), which is expressed as: ; In the formula, This represents the top surface projection region after the target standard model transformation. This represents the area enclosed by the visible edge outline. This represents the area of ​​the region. The intersection-union ratio is a well-known geometric measurement method and will not be elaborated here.

[0040] After completing the shape coverage matching, since the actual vertical dimension occupied by the material is not yet determined, it is necessary to further derive the space occupancy completion parameters 45 for the obscured portion, thereby restoring the complete volume 46 of the individual material. Specifically, this includes: First, starting from the top surface of the target standard model 43 after shape coverage matching 44, the probe is moved downwards along the vertical axis of the three-dimensional coordinate system to the bottom support plane or the top surface of the lower material. Specifically, the depth values ​​in the spatial depth image are sampled layer by layer along the vertical axis. When the bottom support plane, i.e., the plane where the bottom plate of the storage area is located, or the top surface of the lower material, i.e., the highest depth section of the adjacent material below, is detected, the probe stops and the height range traversed during the probe is recorded.

[0041] Secondly, calculate the vertical displacement and use it as the spatial occupancy completion parameter for the obscured portion of the material. The spatial occupancy completion parameter refers to the displacement of the target material from its visible top surface along the vertical direction to the supporting surface, used to characterize the vertical occupancy of the obscured portion. Its calculation expression is as follows: ; In the formula, To fill in the parameters for space placeholders, This represents the vertical coordinates of the top surface of the target standard model in the three-dimensional coordinate system. This indicates the vertical coordinates of the detected bottom support plane or the top surface of the lower material layer.

[0042] Third, based on the bottom surface shape and spatial occupancy completion parameters 45 of the target standard model 43, the physical form of the occluded part is derived and combined with the visible edge contour 41 to form the complete occupied volume 46. Specifically, the bottom surface shape of the target standard model is arranged vertically according to the spatial occupancy completion parameters. Stretching or matching is performed to obtain the three-dimensional shape of the occluded part, and this shape is combined with the upper shape enclosed by the visible edge contour in the three-dimensional coordinate system to obtain the complete volume occupied by the material.

[0043] Through the above fitting and completion operations, the complete three-dimensional shape of individual materials can be reasonably reconstructed even with only the visible outline. Since the complete volumes occupied by different materials in the stacked scene are spatially intertwined, it is necessary to further analyze the spatial relationships of these volumes to identify the stacking levels and covered materials, thus proceeding to the next step.

[0044] Step S14: Determine the stacking level of individual materials based on the physical shielding order of the complete occupied volume in the vertical direction. The stacking level is used to count the actual number of covered materials in the warehouse.

[0045] In this embodiment, after obtaining the complete occupied volume of each material individual, it is necessary to further analyze the relative positions of these occupied volumes in space in order to distinguish material individuals at different stacking levels and identify those bottom materials that are covered by upper materials and are not visible in the surface texture image.

[0046] See Figure 5 As shown, Figure 5 The flowchart illustrates the processing relationships between the complete occupied volume set 51, horizontal plane overlapping projection area calculation 52, volume centroid height extraction 53, vertical occlusion sequence and stacking hierarchy division 54, viewpoint exposure ratio analysis 55, and actual inventory quantity statistics 56. Specifically, this step may include the following sub-steps: Calculate the overlapping projected area 52 of the complete occupied volume set 51 of each material individual on the horizontal plane of the three-dimensional coordinate system. Specifically, project each complete occupied volume onto the horizontal plane along the vertical axis to obtain the corresponding two-dimensional projected area, and perform an intersection operation on the projected areas between each pair of materials to obtain the overlapping projected area. This area is used to measure the degree of mutual coverage between the two materials in the horizontal direction, and its calculation expression is as follows: ; In the formula, and They represent the first The and the first The complete volume occupied by each individual material. This represents the projection operator along the vertical axis to the horizontal plane. This represents the overlapping area of ​​the projected regions of the two materials.

[0047] For overlapping projected area For individual materials exceeding the preset range, extract the volumetric centroid height of each individual material. When the overlapping projected area of ​​two or more materials exceeds the preset range, they are considered to have a horizontal covering relationship and may be in a stacked state. The preset range can be set according to the proportion of the material's projected area, for example, a certain percentage of the smaller material's projected area, specifically based on the material's storage density and category. The volumetric centroid height refers to the coordinate value of the geometric centroid of the fully occupied volume on the vertical axis, and its calculation expression is: ; In the formula, Indicates the height of the center of gravity of the volume. This represents the complete volume area occupied by an individual resource. This represents the coordinates of the volume element along the vertical axis. Let represent a volume element. The above integral can be implemented in actual calculations by summing discrete voxels; specific numerical values ​​will not be substituted into the calculation here.

[0048] According to the height of the center of gravity A vertical shading sequence 54 is established in descending order of size. This vertical shading sequence is used to determine the stacking level to which each individual material belongs. Specifically, the material with the largest center of gravity height is determined to be the top layer, i.e., the first layer. The rest are assigned to the second layer, third layer, and other non-first layers according to their center of gravity height, thus forming a sequence describing the vertical shading relationship of the materials.

[0049] After completing the stacking hierarchy, to avoid missing any covered materials at the bottom layer, it is necessary to further count the actual inventory quantity of covered materials based on the vertical shading sequence. Specifically, this includes: First, identify the bottom-level materials that are not in the top-level stack. The bottom-level stack is any layer other than the top layer, where other materials cover it.

[0050] Secondly, the viewpoint exposure ratio of individual underlying materials in the surface texture image is analyzed. The viewpoint exposure ratio refers to the ratio of the number of visible pixels actually captured in the surface texture image to the theoretical number of pixels projected onto the image plane from the total volume occupied by the underlying material. This ratio measures the degree of occlusion of the material from the current viewpoint, and its calculation expression is as follows: ; In the formula, Indicates the viewpoint exposure ratio. This indicates the number of pixels actually visible in the surface texture image for an individual underlying material. This represents the total number of pixels that theoretically project the complete volume of the material onto the image plane. The total number of theoretically projected pixels is obtained by mapping the complete volume onto the image model.

[0051] Third, if the viewpoint exposure ratio E is lower than the preset display extreme value, the physical existence of individual materials at the bottom layer is determined based on the vertical occlusion sequence, and the total number of materials recorded in all stacking levels (56) is counted to obtain the actual quantity in stock. The preset display extreme value is used to determine whether materials are significantly occluded, and it can be set according to the type of materials and the density of their placement. When the viewpoint exposure ratio is lower than this extreme value, although the materials cannot be directly observed in the image, their physical existence can be reliably confirmed based on the completion results in the aforementioned stacking levels and the vertical occlusion sequence. The visible materials identified in all stacking levels and the covered materials confirmed to exist based on the vertical occlusion sequence are included in the total, thus obtaining the actual quantity of emergency supplies in stock within the storage area.

[0052] By using the above-mentioned stacking hierarchy determination and viewpoint exposure ratio analysis, the problem of missed detection of underlying materials by simple image recognition solutions can be solved, enabling accurate inventory quantities to be obtained even in scenarios with densely stacked materials. Since the static actual inventory quantity cannot directly reflect the supply and demand status of materials in the future, the next step is to perform spatiotemporal correlation analysis between the actual inventory quantity and the predicted consumption data of emergency rescue missions.

[0053] Step S15: Spatiotemporally correlate the actual inventory quantity with the consumption forecast data of related emergency rescue missions, analyze the dynamic matching deviation between the evolution trend of inventory quantity and the consumption forecast demand, and identify target materials in a state of supply imbalance.

[0054] In this embodiment, after obtaining the actual inventory quantity, relying solely on the static quantity cannot guide the dynamic allocation of emergency supplies. Therefore, it is necessary to spatially and temporally correlate the actual inventory quantity with the consumption prediction data of emergency missions, thereby revealing the dynamic matching between inventory levels and mission requirements.

[0055] Specifically, the process of spatiotemporally correlating the actual inventory quantity with the consumption prediction data of related emergency rescue tasks includes: obtaining the historical consumption frequency of related emergency rescue tasks and mapping the historical consumption frequency to a preset future task period. Here, related emergency rescue tasks refer to historical task records corresponding to the emergency rescue areas supported by the current storage area, such as pre-hospital emergency dispatches and disaster emergency rescue; historical consumption frequency refers to the number or quantity of each category of supplies consumed per unit of time. By mapping the historical consumption frequency to a preset future task period according to a time scale, such as by day or by week, a preliminary demand baseline can be formed within that period.

[0056] Based on the urgency of the current emergency response mission, the consumption growth rate of each type of material is determined over the future mission cycle. This consumption growth rate is used to generate consumption forecast data describing the demand at different time points. The consumption growth rate refers to the percentage increase in material consumption relative to a historical baseline as the urgency level increases. The urgency level can be provided by the emergency response mission issuance system in the form of levels, and corresponding rate parameters can be set accordingly, specifically based on the actual mission type and emergency level. The demand rate of the consumption forecast data at time τ can be expressed as: ; In the formula, Let τ be the predicted demand rate. The baseline demand rate is obtained by mapping historical consumption frequency. The consumption growth rate is determined by the urgency of the current emergency response mission.

[0057] See Figure 6 As shown, Figure 6The distribution of the inventory reduction trend line 61 and the demand growth trend line 62 on the same time axis is illustrated in schematic form, along with the spatiotemporal deviation between them 63, and the bottoming point 64 of the inventory reduction trend line within the preset replenishment cycle is marked. After obtaining the consumption forecast data, the dynamic matching deviation between the evolution trend of inventory quantity and the consumption forecast demand is further analyzed to identify target materials in a state of supply imbalance. Specifically, this includes: First, a trend line 61 for the decrease in inventory over time is established based on the actual inventory quantity, and a corresponding demand growth trend line 62 is established based on consumption forecast data. The trend line for the decrease in inventory quantity can be formed by the decrease in inventory quantity over time according to the consumption forecast rate, and their relationship can be described by a function expression of the following form: ; In the formula, Indicates time Inventory balance, This indicates the actual quantity in stock at the initial moment. This represents the rate of time consumption per unit time at time τ. Indicates from time zero to The integral of the accumulated consumption over time. Correspondingly, the demand growth trend line can be formed by accumulating consumption forecast data along the time axis, and its relationship can be described by a function expression of the following form: ; In the formula, Indicates time The cumulative demand, Indicates time The predicted demand rate. The above integral relationship can be discretely summed at equal time intervals in actual calculations. Here, specific values ​​are not substituted into the calculation, but are only used to illustrate the relationship between the reduction of surplus and the evolution of demand growth over time.

[0058] Secondly, calculate the spatiotemporal deviation 63 between the residual decrease trend line 61 and the demand increase trend line 62. The spatiotemporal deviation measures the difference between residual and demand on the same time axis, and can be measured by the difference sequence of the two trend lines at each time node. Its expression is: ; In the formula, Indicates time The degree of spatiotemporal deviation, when Continuously increasing and When the value approaches zero, it indicates that the remaining capacity is no longer sufficient to match the growth in demand.

[0059] Third, if the spatiotemporal deviation indicates that the remaining stock reduction trend line falls below the preset inventory threshold within the preset replenishment period, the corresponding material is determined to be in a state of supply imbalance. In other words, if the spatiotemporal deviation 63 shows that the remaining stock reduction trend line 61 will bottom out at point 64 within the preset replenishment period, the corresponding material is determined to be in a state of supply imbalance. Here, the preset replenishment period refers to the time window required from issuing a replenishment order to the actual arrival of the material; the remaining stock reduction trend line bottoming out means that the trend line will drop to zero or below the safety stock threshold within the preset replenishment period. When the determination condition is met, the corresponding category of material is identified as the target material in a state of supply imbalance.

[0060] Through the aforementioned spatiotemporal correlation analysis, static inventory data can be transformed into dynamic matching results with predictive capabilities, enabling timely detection of potential material shortage risks. After identifying the target materials in a state of supply imbalance, in order to drive subsequent resource allocation, the shortage information of the target materials needs to be transformed into executable replenishment instructions, thus proceeding to the next step.

[0061] Step S16: Extract the missing information of the target materials to generate a material replenishment signal and send it to the management terminal for synchronous updating of the digital material dashboard.

[0062] In this embodiment, after identifying the target materials that are in a state of supply imbalance, it is necessary to convert the shortage information of these target materials into executable replenishment instructions to drive the subsequent resource allocation process.

[0063] Specifically, before generating a supply replenishment signal by extracting information on missing target materials, the process further includes: Obtain real-time task attribute information for the associated emergency rescue area. This real-time task attribute information refers to attribute parameters that reflect the immediate task status, such as task type, task scale, task priority, and the number of rescue personnel deployed, all relevant to the current emergency rescue area. Then, perform a demand coupling analysis by combining the shortage information of target supplies with the real-time task attribute information. This demand coupling analysis is used to determine the urgency level of each target supply for the current task. The demand coupling analysis uses a weighted scoring method; specifically, a comprehensive urgency score is calculated for each target supply, and the calculation expression is: ; In the formula, This represents the overall urgency score of the k-th type of target material. , , to These represent the quantitative scores for the material in terms of factors such as the number of missing items, task priority, task scale, and personnel input. , , to The weighting coefficients correspond to the influencing factors, with a sum of weight values ​​of 1. Specific weights can be set according to the actual task type and management needs. Based on the overall urgency score, each target resource is divided into different supply urgency levels; the higher the score, the higher the supply urgency level.

[0064] Based on the urgency level of supplies, the hierarchical response order for each target material in resource allocation and circulation is determined, and this hierarchical response order serves as a decision-guiding parameter for material supply signals. The hierarchical response order is arranged from highest to lowest urgency level, with the target material with the highest urgency level responding first.

[0065] After establishing the urgency level and tiered response sequence, the shortage information of target materials, the urgency level of replenishment, and the tiered response sequence are packaged into a material replenishment signal and sent to the management terminal. Upon receiving the material replenishment signal, the management terminal executes the replenishment scheduling in sequence according to the decision guidance parameters contained therein, and synchronizes the latest inventory quantity, target material information, and replenishment status to the digital material dashboard, thereby achieving real-time visualization of the status of emergency supplies.

[0066] Therefore, in this application, surface texture images and spatial depth images are first acquired simultaneously using sensing devices, and spatial position benchmarks for individual materials are established in a three-dimensional coordinate system; a deep learning model is used to identify boundary edge lines and locate visible edge contours by combining depth numerical jumps; the complete occupied volume of individual materials is restored by fitting standard specification models and deriving spatial occupancy completion parameters; the stacking level is determined based on the vertical occupancy order of the complete occupied volume, and the actual number of covered materials in the warehouse is counted by combining the viewpoint exposure ratio; furthermore, the actual number in the warehouse is spatiotemporally correlated with consumption prediction data to identify target materials in a state of supply imbalance; and material replenishment signals are generated by combining real-time task attribute information and the digital material dashboard is updated synchronously.

[0067] This approach addresses the issues present in existing solutions, significantly enhances the ability to identify and replenish obscured emergency supplies in stacked storage scenarios, and enables dynamic matching between actual stock levels and predicted consumption needs. This, in turn, improves the accuracy of emergency supplies inventory and the timeliness of replenishment decisions.

[0068] See Figure 7 As shown, Figure 7The connection relationships and data flow of each module in a deep learning-based emergency supplies image recognition and inventory system are illustrated in the form of a structural diagram. The module 71, recognition module 72, derivation module 73, statistics module 74, analysis module 75, and generation module 76 are sequentially connected, forming a complete processing link from image acquisition to replenishment signal generation. This application also discloses a corresponding deep learning-based emergency supplies image recognition and inventory system, including: Module 71 is used to synchronously acquire surface texture images and spatial depth images of the storage area using sensing devices, and to establish a three-dimensional coordinate system for the material storage space. The three-dimensional coordinate system is used to establish the spatial position reference of individual materials. The recognition module 72 is used to input the surface texture image and spatial depth image into the deep learning model, identify the boundary edge line generated by mutual occlusion of materials under the spatial position reference, and locate the visible edge contour of each material individual by combining the depth value jump. The derivation module 73 is used to fit the visible edge contour with the standard specification model using a deep learning model to derive the space occupancy completion parameters of the occupancy part. The space occupancy completion parameters are used to restore the complete volume occupied by the material. The statistics module 74 is used to determine the stacking level of individual materials based on the physical obscuring order of the complete occupied volume in the vertical direction. The stacking level is used to count the actual number of covered materials in the warehouse. Analysis module 75 is used to correlate the actual inventory quantity with the consumption forecast data of related emergency rescue missions in time and space, analyze the dynamic matching deviation between the evolution trend of inventory quantity and the consumption forecast demand, and identify target materials in a state of supply imbalance. The generation module 76 is used to extract the missing information of the target materials, generate the material replenishment signal, and send it to the management terminal for synchronous updating of the digital material dashboard.

[0069] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0070] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0071] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The technical solutions provided by this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for image recognition and inventory of emergency supplies based on deep learning, characterized in that, include: By using sensing devices to simultaneously acquire surface texture images and spatial depth images of the storage area, a three-dimensional coordinate system for the material storage space is established. This three-dimensional coordinate system is used to establish the spatial position reference for individual materials. The surface texture image and spatial depth image are input into a deep learning model to identify the boundary edge lines caused by mutual occlusion of materials under the spatial position reference, and the visible edge contours of each material individual are located by combining the depth value jump. The visible edge contour is fitted to a standard specification model using the deep learning model to derive the spatial occupancy completion parameters of the occupant. These spatial occupancy completion parameters are used to restore the complete volume occupied by the material. Based on the physical shielding order of the complete occupied volume in the vertical direction, the stacking level of the material is determined, and the stacking level is used to count the actual number of covered materials in the warehouse. The actual inventory quantity is spatiotemporally correlated with the consumption prediction data of related emergency rescue missions. The dynamic matching deviation between the evolution trend of the inventory quantity and the consumption prediction demand is analyzed to identify target materials in a state of supply imbalance. Extract the missing information of the target materials to generate a material replenishment signal and send it to the management terminal for synchronous updating of the digital material dashboard.

2. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, Identify the boundary lines caused by mutual occlusion of materials under the spatial location reference, and locate the visible edge contours of each individual material by combining depth numerical jumps, including: Identify the locations where the continuity of pixel attributes is interrupted in the surface texture image to extract preliminary boundary edges; The preliminary boundary edge is mapped onto the spatial depth image, and planar interference textures are removed based on the abrupt change in depth values ​​on both sides of the preliminary boundary edge to establish the boundary edge line describing the physical boundary of the material individual. Using the boundary line as the boundary, a closed region with depth values ​​in the same continuous interval is extracted under the spatial location reference, which serves as the visible edge contour of each individual material.

3. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, The visible edge contour is fitted to a standard specification model using the deep learning model, including: Based on the geometric shape of the visible edge contour of each individual material, the target standard model corresponding to the geometric shape is retrieved from the pre-stored material database. Adjust the rotation angle and scaling ratio of the target standard model in the three-dimensional coordinate system until the top projection shape of the target standard model under the spatial position reference achieves shape coverage matching with the visible edge contour.

4. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, The spatial occupancy completion parameters for the obscured portion are derived. These parameters are used to restore the complete volume occupied by an individual material, including: Starting from the top surface of the target standard model after shape coverage matching, probe downwards along the vertical axis of the three-dimensional coordinate system to the bottom support plane or the top surface of the lower material. Calculate the extension displacement along the vertical axis and use the extension displacement as the spatial occupancy completion parameter for the obscured part of the material individual; Based on the bottom shape of the target standard model and the space occupancy completion parameters, the physical form of the occupied part is derived and combined with the visible edge contour to form the complete occupied volume.

5. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, Based on the physical shielding order of the complete occupied volume in the vertical direction, the stacking level of individual materials is determined, including: Calculate the overlapping projected area of ​​the complete occupied volume of each material individual on the horizontal plane of the three-dimensional coordinate system; For individual materials whose overlapping projection area exceeds a preset range, the volumetric center of gravity height of each individual material is extracted; A vertical shielding sequence is established according to the volume center height from largest to smallest. The vertical shielding sequence is used to determine the stacking level to which each material belongs.

6. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, The actual quantity of covered materials in stock is counted, including: Identify the bottom-level material individuals that are not in the first-order level of the stacking hierarchy; Analyze the viewpoint exposure ratio of the underlying material individuals in the surface texture image; If the viewpoint exposure ratio is lower than the preset display extreme value, the physical existence of the bottom layer material individual is determined based on the vertical occlusion sequence, and the total number of materials recorded in all stacking levels is counted to obtain the actual quantity in stock.

7. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, The actual inventory quantity is spatiotemporally correlated with the consumption prediction data of related emergency rescue tasks, including: Obtain the historical consumption frequency of related emergency rescue tasks, and map the historical consumption frequency to a preset future task cycle; Based on the urgency of the current emergency rescue mission, the consumption growth rate of each type of material is determined within the future mission cycle. This consumption growth rate is used to generate consumption forecast data describing the demand at different time points.

8. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, Analyzing the dynamic mismatch between the evolution trend of inventory levels and predicted consumption demand identifies target materials in a state of supply imbalance, including: Based on the actual inventory quantity, establish a trend line for the decrease in inventory over time, and establish a corresponding trend line for the increase in demand based on the consumption forecast data. Calculate the spatiotemporal deviation between the residual reduction trend line and the demand growth trend line; If the degree of spatiotemporal deviation indicates that the remaining quantity reduction trend line drops below the preset inventory threshold within the preset replenishment cycle, then the corresponding material is determined to be in a state of supply imbalance.

9. The method for image recognition and inventory of emergency supplies based on deep learning according to claim 1, characterized in that, Before generating a material replenishment signal by extracting the shortage information of the target material, the process also includes: Obtain real-time task attribute information for the associated emergency medical area; The shortage information of the target materials is combined with the real-time task attribute information to perform demand coupling analysis. The demand coupling analysis is used to determine the urgency level of each target material for the current task. Based on the supply urgency level, the hierarchical response order of each target material in the resource allocation and circulation is determined, and the hierarchical response order is used as the decision guidance parameter for the material supply signal.

10. A deep learning-based image recognition and inventory system for emergency medical supplies, characterized in that, include: A module is established to synchronously acquire surface texture images and spatial depth images of the storage area using sensing devices, and to establish a three-dimensional coordinate system for the material storage space. The three-dimensional coordinate system is used to establish the spatial position reference of individual materials. The recognition module is used to input the surface texture image and spatial depth image into a deep learning model, identify the boundary edge line generated by mutual occlusion of materials under the spatial position reference, and locate the visible edge contour of each material individual by combining the depth value jump. The derivation module is used to fit the visible edge contour with the standard specification model using the deep learning model, and derive the spatial occupancy completion parameters of the occupant. The spatial occupancy completion parameters are used to restore the complete volume occupied by the material. The statistics module is used to determine the stacking level of individual materials based on the physical obscuring order of the complete occupied volume in the vertical direction. The stacking level is used to count the actual number of covered materials in the warehouse. The analysis module is used to spatially and temporally correlate the actual inventory quantity with the consumption prediction data of related emergency rescue missions, analyze the dynamic matching deviation between the evolution trend of inventory quantity and the consumption prediction demand, and identify target materials in a state of supply imbalance. The generation module is used to extract the missing information of the target materials, generate a material replenishment signal, and send it to the management terminal for synchronously updating the digital material dashboard.

Images