Material management method, device, equipment, medium and product

Abstract

The application discloses a material management method, device, equipment, medium and product, through complementary perception of millimeter wave radar and visual sensor, the material recognition rate in various complex scenes such as dim light and shielding is improved, a nonlinear weight mapping function dynamically adjusts millimeter wave radar weight and visual sensor weight according to real-time environmental parameters, and the dynamic adjustment mechanism ensures that the most suitable sensor data can be used for material identification in complex environments. Through time and space synchronization processing, the consistency of radar point cloud data and image feature data in time and space coordinates is ensured, fusion errors caused by time difference are avoided, the data correction workload in subsequent processing steps is reduced, and the overall processing efficiency is improved. Through fusion confidence, the borrowing and returning strategy of target material can be determined, the occurrence of misoperation is avoided, the accuracy and efficiency of material management are improved, and the high-frequency borrowing and returning demand of material in different environments is met.

Description

Technical Field

[0001] The present application relates to the field of material management technology, and in particular to material management methods, devices, equipment, media and products. Background Art

[0002] With the rapid expansion of server system and board-level testing, the materials being tested (such as server motherboards, hard drives, and network cards) are characterized by a wide variety, large quantity, and rapid turnover. Traditional manual registration methods require checking model numbers, serial numbers, and quantities during the material check-in and check-out process. This is not only inefficient but also prone to omissions, errors, and delayed updates. This leads to chronic discrepancies between bookkeeping records and actual inventory, resulting in generally low inventory accuracy rates for enterprises. This discrepancy directly leads to duplicate purchases, test scheduling conflicts, and asset loss, becoming a bottleneck hindering the efficient operation of test centers.

[0003] To address the drawbacks of manual registration, related technologies use an automatic recognition solution based on QR code scanning. However, the QR code scanning method relies on manual alignment and scanning. In high-frequency borrowing and returning scenarios, the average single operation takes more than 5 seconds, which is inefficient. In addition, there are errors caused by human factors such as missed scans and repeated scans. Therefore, it cannot adapt to high-frequency borrowing and returning scenarios.

[0004] Therefore, how to make test materials adapt to high-frequency borrowing and returning scenarios is an urgent problem that needs to be solved. Summary of the Invention

[0005] The present application provides a material management method, device, equipment, medium and product to at least solve the problem that the material management method of the related art cannot adapt to high-frequency borrowing and returning scenarios.

[0006] This application provides a material management method, including:

[0007] In response to the wake-up interrupt signal, radar perception data and visual perception data of the target material are acquired; the radar perception data includes radar point cloud data, and the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data; the visual perception data includes image feature data, and regional illumination intensity, texture complexity, and contour clarity extracted from the image feature data;

[0008] Calculating a weight of the radar perception data using a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity, and calculating a weight of the visual perception data using a nonlinear weight mapping function according to the contour clarity, the motion speed, and the acceleration;

[0009] Performing time synchronization processing and space alignment processing on the radar point cloud data and the image feature data to generate time-space aligned fusion data;

[0010] Calculating a fusion confidence based on the fusion data, the environmental uncertainty factor, and the motion uncertainty factor, wherein the fusion confidence is used to indicate the credibility of the perception data of the target material;

[0011] A borrowing and returning strategy corresponding to the target material is determined according to the fusion confidence.

[0012] This application also provides a material management device, including:

[0013] a data acquisition module, configured to acquire radar perception data and visual perception data of the target material in response to a wake-up interrupt signal; the radar perception data including radar point cloud data and the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data; and the visual perception data including image feature data and regional illumination intensity, texture complexity, and contour clarity extracted from the image feature data;

[0014] a weight calculation module, configured to calculate the weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity, and the texture complexity through a nonlinear weight mapping function, and to calculate the weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function;

[0015] a synchronization processing module, configured to perform time synchronization processing and spatial alignment processing on the radar point cloud data and the image feature data to generate fused data after time and space alignment;

[0016] a confidence calculation module, configured to calculate a fusion confidence based on the fusion data, an environmental uncertainty factor, and a motion uncertainty factor, wherein the fusion confidence is used to represent the credibility of the perception data of the target material;

[0017] A strategy determination module is used to determine the borrowing and returning strategy corresponding to the target material according to the fusion confidence.

[0018] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned material management methods when executing the computer program.

[0019] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned material management methods are implemented.

[0020] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned material management methods when executed by a processor.

[0021] This application utilizes the complementary sensing capabilities of millimeter-wave radar (penetration) and visual sensors (high resolution) to improve material recognition in complex scenarios, including low light conditions, obscured light, reflective surfaces, and packaging. Furthermore, a nonlinear weight mapping function dynamically adjusts the weights of the millimeter-wave radar and visual sensors based on real-time environmental parameters (light intensity, texture complexity, and packaging thickness). For example, in low-light conditions, the visual sensor weight decreases while the millimeter-wave radar weight increases; in brightly lit conditions, the visual sensor weight increases while the millimeter-wave radar weight decreases. This dynamic adjustment mechanism ensures that the system consistently utilizes the most appropriate sensor data for material recognition in complex and changing environments, significantly improving recognition robustness. Time synchronization ensures temporal consistency between radar point cloud data and image feature data, avoiding fusion errors caused by time differences. Spatial alignment ensures data consistency in spatial coordinates, enabling accurate alignment of the radar point cloud and image features within a unified coordinate system. This reduces the need for data correction and compensation in subsequent processing steps and improves overall processing efficiency. By integrating confidence, the borrowing and returning strategies for target materials can be determined to avoid misoperations, improve the accuracy and efficiency of material management, and meet the high-frequency borrowing and returning needs of materials in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A schematic diagram of a material management method provided in an embodiment of the present application;

[0025] Figure 2 A structural diagram of a material management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] Explanation of terms:

[0030] TSN: Time-Sensitive Networking, time-sensitive network, ensures real-time data transmission.

[0031] PTP: Precision Time Protocol, used for sub-microsecond clock synchronization.

[0032] SHA3-512: Secure Hash Algorithm 3, a cryptographic algorithm used to generate 512-bit hash values.

[0033] ISAC: Integrated Sensing and Communication, communication-perception integrated technology.

[0034] Multimodal fusion: Millimeter-wave radar and visual sensor data collaborate to identify materials, improving environmental adaptability through dynamic weight allocation.

[0035] Sensorless wake-up: The system is automatically triggered by radar micro-motion detection, without the need for active human operation (such as scanning a code).

[0036] Environmental fingerprint chain: A continuous hash sequence generated by environmental parameters (light L / thickness T / texture complexity C / contour clarity S / motion speed V / acceleration a), used for tamper-proof evidence storage on the blockchain.

[0037] Dynamic uncertainty factor: including environmental uncertainty factor and motion uncertainty factor, used to characterize the variable of environmental / motion disturbance, which can be used to correct the fusion confidence.

[0038] Sub-pixel alignment: spatial synchronization accuracy is higher than the size of a single pixel (such as pixel error <0.5 pixels).

[0039] With the rapid expansion of server whole machine and board level test scale, the materials to be tested (such as server mainboard, hard disk, network card, etc.) show the typical characteristics of multiple types, large quantity and fast turnover. The traditional manual registration method needs to check the model, serial number and quantity one by one in the process of material borrowing and returning, which is not only low in efficiency, but also prone to problems such as missing registration, wrong registration and delayed update, resulting in a long-term inconsistency between the account records and the actual inventory, and the inventory accuracy of enterprises is generally low. This inconsistency between account records and actual inventory directly leads to problems such as repeated procurement, test scheduling conflicts and asset loss, which has become a bottleneck restricting the efficient operation of the test center.

[0040] To solve the drawbacks of manual registration, in the related art, automatic identification schemes such as RFID (Radio Frequency Identification), two-dimensional code scanning and machine vision are used, but none of them can fully meet the special needs of the test scene. Among them, the RFID technology is limited by the radio frequency penetration ability, and for the test materials still retaining the anti-static bag, buffer foam or metal shielding layer packaging, the recognition success rate is less than 30%, which cannot complete the identification of the test materials without unpacking. Although the two-dimensional code scanning has low cost, it relies on manual alignment for code scanning, and in the high-frequency borrowing and returning scene, the average single operation time is more than 5 seconds, and there are errors caused by human reasons such as missing scanning and re-scanning, so it cannot adapt to the high-frequency borrowing and returning scene. In addition, the traditional 2D visual identification has a sharp drop in recognition rate to less than 40% in the environment of server rack, dark warehouse or material stacking shielding, while 3D vision can partially alleviate the shielding problem, but the cost of a single set of hardware is more than 20,000 yuan, and the algorithm processing delay is more than 120 milliseconds, which is difficult to meet the real-time requirement of less than 50 milliseconds of the test production line. In summary, the related art has the problems of insufficient recognition penetration, high dependence on manual operation, poor real-time performance and high cost in the test material management scene of high density, high frequency and multiple shielding. Therefore, how to make the test materials adapt to the high-frequency borrowing and returning scene is a problem to be solved at present.

[0041] Embodiments of the present application provide a material management method, which is applied to a material management method, and the method is described in detail in combination with the execution process of the material management method.

[0042] Referring to Figure 1 , as shown, Figure 1This is a flow chart of a material management method provided in an embodiment of the present application. The material management method provided in an embodiment of the present invention includes the following steps:

[0043] S11. In response to the wake-up interrupt signal, obtain radar perception data and visual perception data of the target material.

[0044] The radar perception data includes radar point cloud data, as well as the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data. The visual perception data includes image feature data, as well as regional lighting intensity, texture complexity, and contour clarity extracted from the image feature data.

[0045] Specifically, when the material management system detects a target material entering the monitoring area, the system automatically triggers a wake-up interrupt signal to acquire radar and visual perception data of the target material. Radar perception data includes radar point cloud data, as well as the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data. Visual perception data includes image feature data, as well as regional lighting intensity, texture complexity, and contour clarity extracted from the image feature data. This wake-up interrupt signal is used to initiate the data acquisition process, ensuring that the system only collects data when needed, thereby saving resources and improving response time.

[0046] Millimeter-wave radar sensors acquire radar point cloud data of target materials. This data includes the target material's three-dimensional coordinates and can directly penetrate the outer packaging to obtain internal structural information. The radar point cloud data also extracts the outer packaging's physical thickness, velocity, and acceleration. These parameters are crucial for material identification and classification, especially in dynamic environments.

[0047] It should be noted that the outer packaging physical thickness refers to the outer packaging physical thickness of the target material, which is represented by T and is measured in centimeters. For example, the thicker the outer packaging physical thickness, the worse the visual penetration. In this case, the visual sensor weight needs to be reduced. The motion speed refers to the moving speed of the target material during the operation process, which is represented by V and is measured in meters per second. For example, the faster the motion speed, the more severe the visual motion blur. In this case, the visual weight needs to be reduced. The acceleration refers to the acceleration of the target material movement, which is represented by a and is measured in meters per second. 2 ,Sudden changes in acceleration (such as sudden stops) cause jitter in the radar point cloud, and the influence of the velocity factor needs to be suppressed.

[0048] A visual sensor (such as a high-resolution camera) acquires image feature data of the target material area. This data includes pixel values, which reflect the surface characteristics of the material. Regional lighting intensity, texture complexity, and outline clarity are extracted from this image feature data. These parameters help assess image quality and provide important information for subsequent image processing and analysis.

[0049] It should be noted that regional illumination intensity refers to the ambient lighting in the target material storage / handling area, represented by L and measured in lux. Regional illumination intensity directly impacts visual recognition, so the radar weight should be increased in low-light environments. Texture complexity refers to the complexity of the target material's surface texture, represented by C and calculated using image information entropy in bits. More complex textures, such as motherboard circuits, require a higher visual weight. Outline clarity refers to the sharpness of the target material's edges, represented by S and calculated using the image gradient modulus. Blurred outlines (such as reflections / motion artifacts) indicate lower visual reliability and require a lower visual weight.

[0050] Two of the aforementioned environmental factors require dual sensor acquisition. The first is the outer packaging's physical thickness, T, using a millimeter-wave radar as the primary sensor and a visual sensor as the secondary sensor. The radar penetrates the packaging to measure distance, while visual sensors verify the material's dimensions and infer thickness. The second is contour clarity, S, using a visual sensor as the primary sensor and a millimeter-wave radar as the secondary sensor. The radar provides spatial reference coordinates, while the visual sensor calculates edge gradients to avoid misjudgments of out-of-focus blur due to purely visual perception.

[0051] S12. Calculate the weight of the radar perception data using a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity, and calculate the weight of the visual perception data using a nonlinear weight mapping function according to the contour clarity, the motion speed, and the acceleration.

[0052] The nonlinear weight mapping function can be a hyperbolic tangent function, represented by tanh(x). In weight mapping, tanh(x) can be used to map environmental factors or other features to a range of [0, 1] as a weight coefficient. This allows for dynamic adjustment of the contribution of different sensor data in the fusion process.

[0053] Specifically, a nonlinear weight mapping function is used to calculate the weight of radar data based on the outer packaging's physical thickness, light intensity, and texture complexity. A nonlinear weight mapping function is also used to calculate the weight of visual data based on contour clarity, motion speed, and acceleration. The calculated radar and visual data weights are then applied to subsequent data processing and analysis to optimize the accuracy of material identification and classification. The radar and visual data weights reflect the reliability and importance of different sensor data in specific environments, enabling the system to more intelligently process and fuse data from different sensors.

[0054] In some embodiments, before executing step S12 (calculating the weight of the radar perception data using a nonlinear weight mapping function based on the physical thickness of the outer packaging, the light intensity, and the texture complexity, and calculating the weight of the visual perception data using a nonlinear weight mapping function based on the contour clarity, the motion speed, and the acceleration), the following steps may also be performed:

[0055] The physical thickness of the outer packaging, the movement speed, the acceleration, the light intensity, the texture complexity and the contour clarity are respectively normalized to obtain the normalized physical thickness of the outer packaging, the normalized movement speed, the normalized acceleration, the normalized light intensity, the normalized texture complexity and the normalized contour clarity.

[0056] Specifically, different environmental factors are normalized according to different normalization formulas to obtain the normalized outer packaging physical thickness, normalized movement speed, normalized acceleration, normalized light intensity, normalized texture complexity and normalized contour clarity.

[0057] For example, referring to Table 1, the normalization formula in Table 1 is used to normalize the various environmental factors, mapping all environmental factors to the [0, 1] interval to eliminate dimensional differences and facilitate weight calculation and integration. All environmental factors are directly related to the real-time state of the test material (such as packaging thickness, motion trajectory, surface texture characteristics, etc.).

[0058] It should be noted that the acceleration does not need to be normalized, and only its absolute value is required. Moreover, the acceleration range is 0-5 m / s. 2 , truncated if outside this range. Furthermore, the key parameters were designed based on the following: 500 Lux illumination: the minimum illumination for material inspection specified in the ISO 3664 standard; 30 cm thickness: the statistical upper limit for the thickness of server motherboard / hard drive packaging; and 5 bits of texture entropy: the classification threshold for the OpenCV test dataset (entropy < 5 indicates simple textures). The parameter baseline values ​​(500 Lux, 30 cm, 5 bits) are derived from actual measured data from server test scenarios and are not theoretical assumptions.

[0059] Table 1

[0060]

[0061] Among them, illumination and thickness are normalized by truncation to control the influence of extreme values, and the inverse tangent transformation / S-type transformation of speed / texture can enhance the robustness of the algorithm. e is a natural constant used to calculate , which is used to construct an exponential decay function and map the gradient modulus S to the interval [0, 1).

[0062] In some embodiments, the above step S12 (calculating the weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity, and the texture complexity through a nonlinear weight mapping function, and calculating the weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function) can be implemented as follows:

[0063] (1) Calculating the initial weight of the radar perception data through a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity.

[0064] Optionally, the above step (1) of calculating the initial weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity and the texture complexity through a nonlinear weight mapping function can be implemented as follows:

[0065] The initial weight of the radar perception data is calculated by a nonlinear weight mapping function based on the normalized physical thickness of the outer packaging, the normalized light intensity, the normalized texture complexity, the first learning coefficient, and the second learning coefficient; wherein the first learning coefficient is a weight coefficient that balances the light intensity and the physical thickness of the outer packaging; and the second learning coefficient is a weight coefficient for texture complexity.

[0066] Specifically, the initial weight of the radar perception data is calculated according to the following formula:

[0067] ;

[0068] in, represents the initial weight of radar perception data, Represents the first learning coefficient, which is used to balance the weight influence of illumination and thickness. The default value can be 0.5. represents the normalized light intensity, Indicates the normalized physical thickness of the outer packaging, Represents the second learning coefficient. The default value can be 0.5. The second learning coefficient is used to control the weight contribution of texture complexity. Indicates the normalized texture complexity. Represents the light suppression item. The darker the environment, the larger the value. Represents the thickness suppression term. The larger the value, the larger the denominator. The illumination suppression term decreases more slowly than the thickness suppression term, so thick packages still tend to be radar-dominated.

[0069] (2) Calculating the initial weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function.

[0070] Optionally, the above step (2) of calculating the initial weight of the visual perception data according to the contour clarity, the motion speed and the acceleration through a nonlinear weight mapping function can be implemented as follows:

[0071] An adaptive speed attenuation coefficient of the target material is calculated according to the normalized motion speed and the normalized acceleration.

[0072] Specifically, it is calculated according to the following formula :

[0073] ;

[0074] in, represents the normalized motion speed, represents the normalized acceleration.

[0075] According to the normalized contour clarity and the adaptive speed attenuation coefficient, the third learning coefficient and the fourth learning coefficient, the initial weight of the visual perception data is calculated by a nonlinear weight mapping function; the third learning coefficient is the contour clarity weight coefficient, and the fourth learning coefficient is the motion stability weight coefficient.

[0076] Specifically, the initial weight of visual perception data is calculated according to the following formula:

[0077] ;

[0078] in, represents the initial weight of visual perception data, Indicates the third learning coefficient, which is the contour clarity weight coefficient. The default value can be 0.6. represents the normalized contour clarity, Indicates the fourth learning coefficient. The default value can be 0.4. The fourth learning coefficient is the motion stability weight coefficient. represents the normalized acceleration.

[0079] (3) Performing exponential normalization processing on the initial weight of the radar perception data to calculate the weight of the radar perception data.

[0080] Specifically, the initial weight of the radar perception data is exponentially normalized according to the following formula to obtain the (final) weight of the radar perception data:

[0081] ;

[0082] in, represents the (final) weight of the radar perception data, represents the initial weight of radar perception data, Represents the initial weight of visual perception data. It is a natural constant used to compress the basic value of the weight to the interval [0, 1] and maintain the proportional relationship.

[0083] (4) Calculating the weight of the visual perception data based on the weight of the radar perception data.

[0084] Specifically, since the sum of the weight of the radar perception data and the weight of the visual perception data is 1, the (final) weight of the visual perception data is calculated according to the following formula:

[0085] ;

[0086] in, represents the (final) weight of the visual perception data, Represents the (final) weight of the radar perception data.

[0087] In the disclosed embodiments, this method dynamically adjusts weights to adapt to varying environmental conditions, enabling accurate material identification and classification in complex environments, significantly improving the efficiency and accuracy of material management. Furthermore, this method supports automated operations, reducing the need for manual intervention and thus lowering operating costs.

[0088] S13. Performing time synchronization processing and space alignment processing on the radar point cloud data and the image feature data to generate time-space aligned fusion data.

[0089] In some embodiments, the above step S13 (performing time synchronization processing and spatial alignment processing on the radar point cloud data and the image feature data to generate time-space aligned fusion data) can be implemented as follows:

[0090] 1) At each sampling moment, the median of the Precision Time Protocol clock source, the Network Time Protocol clock source, and the Global Positioning System clock source is used as the reference time.

[0091] Specifically, at each sampling moment, the median of the Precision Time Protocol clock source, the Network Time Protocol clock source, and the Global Positioning System clock source is used as the reference time, which can eliminate the possibility of anomalies in a single clock source.

[0092] 2) Calculating the variance of the reference time within a preset sliding window, and calculating the clock jitter compensation value according to the variance of the reference time.

[0093] Optionally, the above step 2) (calculating the variance of the reference time within a preset sliding window, and calculating the clock jitter compensation value based on the variance of the reference time) can be implemented as follows:

[0094] Calculating a sliding mean of the reference time within the preset sliding window;

[0095] Calculating a clock jitter variance according to the reference time and a sliding mean of the reference time;

[0096] Calculating an adaptive smoothing factor based on the precision time protocol clock source and the network time protocol clock source;

[0097] A clock jitter compensation value is calculated according to the clock jitter variance and the adaptive smoothing factor.

[0098] Specifically, within a preset sliding window, a sliding mean of the reference time is calculated;

[0099] ;

[0100] in, represents the sliding mean of the reference time, Indicates the The median of the three source clocks of the sub-sample, Indicates the number of samples within the preset sliding window, usually Take 50, which corresponds to 1 second of data.

[0101] The clock jitter variance is calculated using the following formula:

[0102] ;

[0103] in, represents the clock jitter variance, represents the sliding mean of the reference time, Indicates the The median of the three source clocks of the sub-sample, Indicates the number of samples within the preset sliding window, usually Take 50, which corresponds to 1 second of data.

[0104] Calculate the clock jitter compensation value according to the following formula:

[0105] ;

[0106] in, represents the adaptive smoothing factor, Represents the clock jitter variance.

[0107] Optionally, the adaptive smoothing factor is calculated according to the following formula:

[0108] ;

[0109] in, represents the adaptive smoothing factor, , Indicates the absolute time difference between PTP and NTP clock sources. Represents the Precision Time Protocol clock source, Indicates the Network Time Protocol clock source.

[0110] 3) Calculating a corrected synchronization timestamp based on the reference time and the clock jitter compensation value, and correcting the first timestamp of the millimeter-wave radar and the second timestamp of the visual sensor based on the synchronization timestamp so that the synchronization error between the first timestamp and the second timestamp remains within a preset time period.

[0111] Specifically, the corrected synchronization timestamp is calculated according to the following formula:

[0112] ;

[0113] in, Indicates the corrected synchronization timestamp, Represents the Precision Time Protocol clock source, Indicates the Network Time Protocol clock source, Indicates the global positioning system clock source, Indicates taking the median value of the three source clocks. represents the adaptive smoothing factor, Represents the clock jitter variance. Indicates the clock jitter compensation value.

[0114] 4) Obtaining an initial extrinsic parameter matrix of the millimeter-wave radar and the visual sensor, wherein the initial extrinsic parameter matrix includes an initial rotation parameter and an initial translation parameter.

[0115] Specifically, the initial extrinsic parameter matrix of the millimeter wave radar and the visual sensor is obtained, wherein the initial extrinsic parameter matrix includes an initial rotation parameter and an initial translation parameter.

[0116] 5) Construct the objective function based on the weighted sum of the reprojection error and the motion consistency error.

[0117] Among them, the reprojection error is used to represent the square of the distance between the radar point cloud data and the corresponding pixel point after being projected onto the image plane by the initial extrinsic parameter matrix; the motion consistency error is used to represent the square of the directional deviation between the two-dimensional motion vector calculated by the image optical flow and the three-dimensional motion vector measured by the radar Doppler after being projected by the initial extrinsic parameter matrix.

[0118] Optionally, step 5 above (constructing the objective function based on the weighted sum of the reprojection error and the motion consistency error) can be implemented as follows:

[0119] The objective function is constructed according to the following formula:

[0120] ;

[0121] in, represents the coordinates of the image feature points, represents the projection function that maps three-dimensional coordinates to two-dimensional coordinates, represents the camera intrinsic parameter matrix, represents the rotation parameter, represents the radar point cloud coordinates, represents the translation parameter, represents the image gradient field, represents the optical flow vector, represents the radar point motion vector, represents a balance factor, which is used to adjust the weights of the reprojection error term and the motion consistency error term.

[0122] For example, refer to Table 2, which provides the physical meaning, dimension / type, and data source of each parameter in the above objective function. The physical meaning is to calculate the radar point By transforming Position projected onto the image plane, optical flow-radar motion constraint middle, represents visually inferred physical motion, It is used to represent the radar point motion vector corresponding to the real motion measured by the radar. It can be understood that It is used to control the relative contribution of the two types of constraints in the objective function so that the solved [R|t] satisfies both static alignment and dynamic consistency.

[0123] Table 2

[0124]

[0125] In this disclosed embodiment, this objective function is used to determine the optimal spatial transformation parameters (rotation matrix (R) and translation vector (t)) between the millimeter-wave radar and visual sensor, achieving sub-pixel spatial synchronization. Its core purpose is to minimize two types of errors: reprojection error, which ensures the geometric accuracy of the radar point cloud projected onto the image plane; and optical flow-radar motion consistency error, which ensures that the visually inferred motion is consistent with the radar measurement. Ultimately, this objective function achieves the dual goals of static position alignment and dynamic motion matching.

[0126] 6) Iterating the initial extrinsic parameter matrix using a preset algorithm until the reprojection error is less than a preset pixel value, thereby obtaining target rotation parameters and target translation parameters.

[0127] The preset pixel value may be 0.5 pixels or other reasonable values, and is not specifically limited here.

[0128] 7) Mapping the radar point cloud data and the image feature data to a preset spatial coordinate system according to the target rotation parameter and the target translation parameter to generate spatiotemporally aligned fused data.

[0129] Among them, the fusion data includes: the distance from the target material to the millimeter-wave radar, the signal-to-noise ratio of the millimeter-wave radar, the angle between the beam of the millimeter-wave radar and the surface normal of the target material, the overlap between the target material and the detection frame, the image blur of the target material, the classification score of the target material, etc.

[0130] Specifically, the radar point cloud data and the image feature data are mapped to a preset spatial coordinate system according to the target rotation parameter and the target translation parameter to generate fused data after time and space alignment.

[0131] Through the above method, the spatial alignment problem of radar point cloud and visual image can be solved, sub-pixel accuracy can be achieved (error < 2.5 pixels), and multimodal data can be ensured to be spatially consistent.

[0132] Time synchronization ensures temporal consistency between radar point cloud data and image feature data, avoiding fusion errors caused by time differences. Spatial alignment ensures data consistency in spatial coordinates, enabling accurate alignment of radar point cloud and image feature data within a unified coordinate system. Time synchronization and spatial alignment reduce data mismatches caused by sensor errors, environmental changes, and other factors, thereby enhancing system robustness. Precise time synchronization and spatial alignment reduce the need for data correction and compensation in subsequent processing steps, thereby improving the efficiency of accurate and reliable material management in complex environments.

[0133] S14. Calculate fusion confidence based on the fusion data, the environmental uncertainty factor, and the motion uncertainty factor, where the fusion confidence is used to represent the credibility of the perception data of the target material.

[0134] Optionally, the above step S14 (calculating the fusion confidence based on the fusion data, the environmental uncertainty factor, and the motion uncertainty factor, the fusion confidence being used to indicate the credibility of the perception data of the target material) can be implemented as follows:

[0135] An initial radar confidence is calculated based on the distance from the target material to the millimeter-wave radar, the signal-to-noise ratio of the millimeter-wave radar, and the angle between the beam of the millimeter-wave radar and the surface normal of the target material.

[0136] Specifically, the initial radar confidence is calculated according to the following formula:

[0137] ;

[0138] in, shows the initial radar confidence, represents the distance attenuation coefficient, Indicates the straight-line distance from the material to the radar, represents the radar signal-to-noise ratio, represents the signal-to-noise ratio threshold, represents the Sigmoid steepness coefficient, Indicates the angle between the radar beam and the normal of the material surface.

[0139] Obtain the overlap between the target material and the detection frame, the classification score of the target material, and the image blur of the target material.

[0140] An initial visual confidence is calculated based on the degree of overlap between the target material and the detection frame, the classification score of the target material, and the image blur of the target material.

[0141] Specifically, the initial visual confidence is calculated according to the following formula:

[0142] ;

[0143] in, represents the initial visual confidence, Indicates the overlap between the detection frame and the material ROI, Indicates the classification score of the target material, represents the fuzzy suppression coefficient, Indicates the image blur.

[0144] A radar confidence level is calculated based on the initial radar confidence level and the motion uncertainty factor.

[0145] The motion uncertainty factor is a variable used to reflect the decrease in recognition reliability due to the change in the motion state of the target material.

[0146] Specifically, the motion uncertainty factor is calculated according to the following formula:

[0147] ;

[0148] in, represents the motion uncertainty factor, represents a normalized motion speed, represents an acceleration.

[0149] Specifically, the radar confidence is calculated according to the following formula:

[0150]

[0151] wherein, represents an initial radar confidence, represents a motion uncertainty factor.

[0152] According to the initial visual confidence and the environment uncertainty factor, a visual confidence is calculated.

[0153] wherein, the environment uncertainty factor is a variable for reflecting that the visual recognition reliability is reduced due to the external environment change.

[0154] Specifically, the environment uncertainty factor is calculated according to the following formula:

[0155]

[0156] wherein, represents an environment uncertainty factor, represents a normalized contour distinctness, represents a normalized illumination intensity, represents a normalized outer package physical thickness.

[0157] Specifically, the visual confidence is calculated according to the following formula:

[0158]

[0159] wherein, represents an initial visual confidence, represents an environment uncertainty factor.

[0160] According to the weight of the radar perception data, the radar confidence, the weight of the visual perception data, and the visual confidence, an original fusion score is calculated.

[0161] Specifically, the original fusion score is calculated according to the following formula:

[0162]

[0163] wherein, represents an original fusion score, represents a weight of radar perception data, represents a radar confidence, represents a weight of visual perception data, ​​​​represents the visual confidence, represents the environmental uncertainty factor, represents the motion uncertainty factor.

[0164] The original fusion score is subjected to range compression processing according to the inverse tangent function to obtain the fusion confidence.

[0165] Specifically, the fusion confidence is calculated according to the following formula:

[0166] ;

[0167] in, represents the fusion confidence, represents the inverse tangent function, represents the original fusion score.

[0168] It should be noted that the value range of the original fusion score is [0, +∞). The inverse tangent function is used to compress the value range of the original fusion score to the final confidence level in the interval (0, 1), and eliminate the linear growth problem of the denominator.

[0169] In the disclosed embodiments, by integrating radar and visual perception data, the system can verify the presence and characteristics of target materials from different angles and methods, adapting to different environmental conditions and target material characteristics such as distance, signal-to-noise ratio, and beam angle. This not only enhances the system's robustness in various situations, but also optimizes the decision-making process, reduces false positives and false negatives, improves automation, and supports the processing of complex scenarios, thereby improving recognition accuracy. Furthermore, the fusion confidence level provides a quantitative basis for material management decisions, helping the system make more reasonable choices between automation and manual intervention.

[0170] S15. Determine a borrowing and returning strategy corresponding to the target material according to the fusion confidence.

[0171] In some embodiments, the above step S15 (determining the borrowing and returning strategy corresponding to the target material according to the fusion confidence) can be implemented as follows:

[0172] When the fusion confidence falls within the first confidence threshold interval, the borrowing and returning strategy is determined to be controlling the electric lock to release the target material.

[0173] When the fusion confidence falls within the second confidence threshold interval, the borrowing and returning strategy is determined to trigger the millimeter wave radar and the visual sensor to independently identify the target material and perform a logical and cross-check.

[0174] When the fusion confidence falls within a third confidence threshold range, the borrowing and returning strategy is determined to be generating a visual operation guide.

[0175] Among them, the first confidence threshold interval, the second confidence threshold interval and the third confidence threshold interval do not overlap, the center value of the first confidence threshold interval is greater than the center value of the second confidence threshold interval, and the center value of the second confidence threshold interval is greater than the center value of the third confidence threshold interval.

[0176] Specifically, when the fusion confidence falls within the first confidence threshold interval, the system determines the borrowing and returning strategy as automatically releasing the electric lock to allow the target material to be borrowed or returned; this strategy is suitable for situations with very high confidence, which can reduce manual intervention and improve operational efficiency. When the fusion confidence falls within the second confidence threshold interval, the system triggers the millimeter-wave radar and visual sensor to independently identify the target material and perform logical and cross-verification; this strategy is suitable for situations with medium confidence, and improves the accuracy of recognition through cross-validation. When the fusion confidence falls within the third confidence threshold interval, the system determines the borrowing and returning strategy as generating visual operation guidance, such as providing operation guidance to the operator through augmented reality (AR) equipment; this strategy is suitable for situations with low confidence, and ensures the accuracy of the operation through manual assistance.

[0177] Exemplarily, the first confidence threshold interval is [0.8, 1.0], the second confidence threshold interval is [0.65, 0.8), and the third confidence threshold interval is [0, 0.65).

[0178] In the disclosed embodiment, the material management system can adopt appropriate borrowing and returning strategies at different confidence levels, thereby improving the accuracy and efficiency of operations. In addition, the method also supports automated operations and manual assistance, adapting to different operating environments and needs.

[0179] The material management method provided by the disclosed embodiments utilizes the complementary sensing capabilities of millimeter-wave radar (penetration) and visual sensors (high-resolution) to improve material recognition in a variety of complex scenarios, including low light, obscured light, reflective surfaces, and packaging. Furthermore, a nonlinear weight mapping function dynamically adjusts the weights of the millimeter-wave radar and visual sensors based on real-time environmental parameters (light intensity, texture complexity, and packaging thickness). For example, in low light conditions, the visual sensor weight decreases while the millimeter-wave radar weight increases, and vice versa. This dynamic adjustment mechanism ensures that the system consistently utilizes the most appropriate sensor data for material recognition in complex and changing environments, significantly improving recognition robustness. Time synchronization ensures temporal consistency between radar point cloud data and image feature data, avoiding fusion errors caused by time differences. Spatial alignment ensures data consistency in spatial coordinates, enabling accurate alignment of the radar point cloud and image features within a unified coordinate system. This reduces the need for data correction and compensation in subsequent processing steps and improves overall processing efficiency. By integrating confidence, the borrowing and returning strategies for target materials can be determined to avoid misoperations, improve the accuracy and efficiency of material management, and meet the high-frequency borrowing and returning needs of materials in different environments.

[0180] In some embodiments, after executing step S15 (determining the borrowing and returning strategy corresponding to the target material according to the fusion confidence), the following steps may also be executed:

[0181] Taking a first preset time period as a period, obtaining multiple environmental factor values ​​of the current period;

[0182] Normalizing the multiple environmental factor values, and splicing the normalized multiple environmental factor values ​​to obtain spliced ​​environmental parameters;

[0183] Performing a hash operation on the splicing environment parameters to generate an environment fingerprint;

[0184] Acquire multiple environmental fingerprints corresponding to multiple consecutive periods, and generate an environmental fingerprint chain based on the multiple environmental fingerprints;

[0185] When a material borrowing or returning event is triggered, the environmental fingerprint chain is written into the blockchain.

[0186] The first preset duration may be 20 milliseconds, or other reasonable values, which are not specifically limited here.

[0187] Specifically, a plurality of environmental factor values of a current period are obtained periodically with the first preset time length as a period, the plurality of environmental factor values including the area light intensity, texture complexity, contour clarity, outer packaging physical thickness, motion speed and acceleration of the material, which can affect the identification and classification of the material. Then, the plurality of collected environmental factor values are normalized to ensure that data of different dimensions can be effectively compared and fused. The plurality of normalized environmental factor values are spliced to obtain spliced environmental parameters, and the spliced environmental parameters are subjected to hash operation to generate an environmental fingerprint. A plurality of environmental fingerprints corresponding to a plurality of consecutive periods are obtained, and an environmental fingerprint chain is generated according to the plurality of environmental fingerprints. The environmental fingerprint chain can provide a record of the change of the environmental state within a period of time (for example, within one second). When the material lending and returning event is triggered, the environmental fingerprint chain is written into the blockchain. This step uses the blockchain technology to enhance the security and non-tamperability of the data, and provides reliable audit tracking for the material lending and returning event.

[0188] In the embodiments of the present disclosure, by writing the environmental fingerprint chain into the blockchain, it can be ensured that the record of the material lending and returning event has non-tamperability, thereby enhancing the security and traceability of the entire process. The distributed nature of the blockchain ensures the integrity and transparency of the data, so that any attempt to tamper with the record can be detected. The environmental fingerprint chain recorded on the blockchain can be combined with the smart contract to realize the automation of the material lending and returning process.

[0189] In some embodiments, in response to a user input audit event identifier, a target environmental fingerprint chain corresponding to the audit event identifier is obtained; a plurality of environmental factor values corresponding to the audit event identifier are obtained by back calculation according to the target environmental fingerprint chain; and an audit report is generated according to the plurality of environmental factor values corresponding to the audit event identifier.

[0190] Specifically, the system provides a user interface that allows users to enter a specific audit event identifier, which can be a timestamp, a unique event ID, or other attribute that uniquely identifies the event. In response to the user input, the system retrieves the environmental fingerprint chain corresponding to the entered audit event identifier from the blockchain, ensuring data security and immutability. Using the data in the environmental fingerprint chain, the system performs reverse calculations to obtain the multiple environmental factor values ​​corresponding to the audit event identifier. This reverse calculation process involves decrypting hash values ​​and parsing concatenated environmental parameters. The environmental factor values ​​obtained through reverse calculation undergo necessary data processing, such as denoising and format conversion, to ensure data accuracy and usability. Based on the processed environmental factor values, the system automatically generates an audit report containing detailed information about the event, such as the time of occurrence, environmental conditions (environmental factors such as light intensity, texture complexity, and outline clarity), and any other relevant environmental data. After the report is generated, the system also provides a verification step, allowing users or auditors to verify the report's accuracy and completeness.

[0191] Through this method, the material management system can provide a reliable audit mechanism that allows users to track and analyze the environmental conditions of specific events. This mechanism not only improves the transparency and traceability of material management, but also provides important environmental data support for event analysis and decision-making, helping to improve management efficiency and decision-making quality.

[0192] In some embodiments, the system first collects operational data, which may include dual sensor readings (e.g., multiple environmental factors). The collected data is then checked for anomalies. If the test result is normal, the data is archived for future analysis or auditing. If the test result is abnormal, a reinforcement learning optimization process is triggered. After detecting an anomaly, the system performs reinforcement learning optimization, which updates the parameters to improve performance or resolve the detected problem. Finally, the system performs model hot deployment, which applies the new model or parameters to the production environment without stopping service. There are three types of optimization cycles: short-term, which involves online fine-tuning of weight coefficients every 4 hours to quickly adapt to environmental changes; medium-term, which involves updating the noise classification model weekly to improve classification accuracy; and long-term, which involves upgrading the optical flow constraint algorithm quarterly to improve the algorithm's long-term performance and stability.

[0193] Improve performance and reliability through continuous data collection, analysis, and model adjustments. This adaptive optimization mechanism helps increase the intelligence of the system, reduces manual intervention, and ensures that the system can continuously adapt to new challenges and demands.

[0194] In the disclosed embodiments, the complementary sensing capabilities of millimeter-wave radar (penetration) and visual sensors (high resolution) improve material recognition in complex scenarios, including low light conditions, obscured light, reflective surfaces, and packaging. Furthermore, a nonlinear weight mapping function dynamically adjusts the weights of the millimeter-wave radar and visual sensors based on real-time environmental parameters (light intensity, texture complexity, and packaging thickness). For example, in low light conditions, the visual sensor weight decreases while the millimeter-wave radar weight increases, and vice versa. This dynamic adjustment mechanism ensures that the system consistently utilizes the most appropriate sensor data for material recognition in complex and changing environments, significantly improving recognition robustness. Time synchronization ensures temporal consistency between radar point cloud data and image feature data, avoiding fusion errors caused by time differences. Spatial alignment ensures data consistency in spatial coordinates, enabling accurate alignment of the radar point cloud and image features within a unified coordinate system. This reduces the need for data correction and compensation in subsequent processing steps and improves overall processing efficiency. By integrating confidence, the borrowing and returning strategies for target materials can be determined to avoid misoperations, improve the accuracy and efficiency of material management, and meet the high-frequency borrowing and returning needs of materials in different environments.

[0195] Figure 2 This is a structural diagram of a material management device 200 provided by the present disclosure, such as Figure 2 As shown, the device of this embodiment includes:

[0196] The data acquisition module 210 is configured to acquire radar perception data and visual perception data of the target material in response to the wake-up interrupt signal; the radar perception data includes radar point cloud data, and the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data; the visual perception data includes image feature data, and regional illumination intensity, texture complexity, and contour clarity extracted from the image feature data;

[0197] a weight calculation module 220 for calculating the weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity, and the texture complexity using a nonlinear weight mapping function, and calculating the weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration using a nonlinear weight mapping function;

[0198] A synchronization processing module 230 is used to perform time synchronization processing and spatial alignment processing on the radar point cloud data and the image feature data to generate fused data after time and space alignment;

[0199] A confidence calculation module 240 is configured to calculate a fusion confidence based on the fusion data, the environmental uncertainty factor, and the motion uncertainty factor, wherein the fusion confidence is used to indicate the credibility of the perception data of the target material;

[0200] The strategy determination module 250 is used to determine the borrowing and returning strategy corresponding to the target material according to the fusion confidence.

[0201] As an optional implementation of the embodiment of the present disclosure, the policy determination module 250 is specifically configured to:

[0202] When the fusion confidence falls within a first confidence threshold interval, determining the borrowing and returning strategy to be controlling the electric lock to release the target material;

[0203] When the fusion confidence falls within the second confidence threshold range, determining the borrowing and returning strategy to trigger the millimeter-wave radar and the visual sensor to independently identify the target material and perform a logical and cross-check;

[0204] When the fusion confidence falls within a third confidence threshold range, determining that the borrowing and returning strategy is to generate a visual operation guide;

[0205] Among them, the first confidence threshold interval, the second confidence threshold interval and the third confidence threshold interval do not overlap, the center value of the first confidence threshold interval is greater than the center value of the second confidence threshold interval, and the center value of the second confidence threshold interval is greater than the center value of the third confidence threshold interval.

[0206] As an optional implementation of the embodiment of the present disclosure, the weight calculation module 220 includes:

[0207] a radar weight calculation unit, configured to calculate an initial weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity, and the texture complexity through a nonlinear weight mapping function;

[0208] a visual weight calculation unit, configured to calculate an initial weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function;

[0209] an exponential normalization unit, performing exponential normalization processing on the initial weight of the radar perception data to calculate the weight of the radar perception data;

[0210] A dynamic adjustment unit is used to calculate the weight of the visual perception data according to the weight of the radar perception data.

[0211] As an optional implementation of the embodiment of the present disclosure, the device further includes:

[0212] The normalization processing module is used to normalize the physical thickness of the outer packaging, the movement speed, the acceleration, the light intensity, the texture complexity and the contour clarity, respectively, to obtain the normalized physical thickness of the outer packaging, the normalized movement speed, the normalized acceleration, the normalized light intensity, the normalized texture complexity and the normalized contour clarity.

[0213] As an optional implementation of the embodiment of the present disclosure, the radar weight calculation unit is specifically configured to:

[0214] The initial weight of the radar perception data is calculated through a nonlinear weight mapping function based on the normalized physical thickness of the outer packaging, the normalized light intensity, the normalized texture complexity, the first learning coefficient, and the second learning coefficient; wherein the first learning coefficient is a weight coefficient that balances the light intensity and the physical thickness of the outer packaging; and the second learning coefficient is a weight coefficient for texture complexity.

[0215] As an optional implementation of the embodiment of the present disclosure, the visual weight calculation unit is specifically configured to:

[0216] Calculating an adaptive speed attenuation coefficient of the target material according to the normalized motion speed and the normalized acceleration;

[0217] According to the normalized contour clarity and the adaptive speed attenuation coefficient, the third learning coefficient and the fourth learning coefficient, the initial weight of the visual perception data is calculated by a nonlinear weight mapping function; the third learning coefficient is the contour clarity weight coefficient, and the fourth learning coefficient is the motion stability weight coefficient.

[0218] As an optional implementation of the embodiment of the present disclosure, the synchronization processing module 230 includes:

[0219] a reference time calculation unit, configured to use, at each sampling moment, a median of a precision time protocol clock source, a network time protocol clock source, and a global positioning system clock source as a reference time;

[0220] a jitter compensation calculation unit, configured to calculate a variance of the reference time within a preset sliding window, and calculate a clock jitter compensation value according to the variance of the reference time;

[0221] a timestamp synchronization unit, configured to calculate a corrected synchronization timestamp based on the reference time and the clock jitter compensation value, and correct the first timestamp of the millimeter-wave radar and the second timestamp of the visual sensor based on the synchronization timestamp, so that the synchronization error between the first timestamp and the second timestamp remains within a preset time length;

[0222] A parameter acquisition unit, configured to acquire an initial extrinsic parameter matrix of the millimeter-wave radar and the visual sensor, wherein the initial extrinsic parameter matrix includes an initial rotation parameter and an initial translation parameter;

[0223] A function construction unit is configured to construct an objective function based on a weighted sum of a reprojection error and a motion consistency error; the reprojection error is used to represent the square of the distance between the radar point cloud data and the corresponding pixel point after being projected onto the image plane by the initial extrinsic parameter matrix; the motion consistency error is used to represent the square of the directional deviation between the two-dimensional motion vector calculated by the image optical flow and the three-dimensional motion vector measured by the radar Doppler after being projected by the initial extrinsic parameter matrix;

[0224] an iterative calculation unit, configured to iterate the initial extrinsic parameter matrix using a preset algorithm until the reprojection error is less than a preset pixel value, thereby obtaining a target rotation parameter and a target translation parameter;

[0225] A data generation unit is used to map the radar point cloud data and the image feature data to a preset spatial coordinate system according to the target rotation parameter and the target translation parameter, and generate fused data after time and space alignment.

[0226] As an optional implementation of the embodiment of the present disclosure, the jitter compensation calculation unit is specifically configured to:

[0227] Calculating a sliding mean of the reference time within the preset sliding window;

[0228] Calculating a clock jitter variance according to the reference time and a sliding mean of the reference time;

[0229] Calculating an adaptive smoothing factor based on the precision time protocol clock source and the network time protocol clock source;

[0230] A clock jitter compensation value is calculated according to the clock jitter variance and the adaptive smoothing factor.

[0231] As an optional implementation of the embodiment of the present disclosure, the function construction unit is specifically configured to:

[0232] The objective function is constructed according to the following formula:

[0233] ;

[0234] in, represents the coordinates of the image feature points, represents the projection function that maps three-dimensional coordinates to two-dimensional coordinates, represents the camera intrinsic parameter matrix, represents the rotation parameter, represents the radar point cloud coordinates, represents the translation parameter, represents the image gradient field, represents the optical flow vector, represents the radar point motion vector, represents a balance factor, which is used to adjust the weights of the reprojection error term and the motion consistency error term.

[0235] As an optional implementation of the embodiment of the present disclosure, the device further includes a data writing module; the data writing module is specifically configured to:

[0236] Taking a first preset time period as a period, obtaining multiple environmental factor values ​​of the current period;

[0237] Normalizing the multiple environmental factor values, and concatenating the normalized multiple environmental factor values ​​to obtain a concatenated environmental parameter;

[0238] Performing a hash operation on the splicing environment parameters to generate an environment fingerprint;

[0239] Acquire multiple environmental fingerprints corresponding to multiple consecutive periods, and generate an environmental fingerprint chain based on the multiple environmental fingerprints;

[0240] When a material borrowing or returning event is triggered, the environmental fingerprint chain is written into the blockchain.

[0241] As an optional implementation of the embodiment of the present disclosure, the apparatus further includes an audit report generation module, and the audit report generation module is specifically configured to:

[0242] In response to the audit event identifier input by the user, obtaining a target environment fingerprint chain corresponding to the audit event identifier;

[0243] Perform reverse calculation based on the target environment fingerprint chain to obtain multiple environmental factor values ​​corresponding to the audit event identifier;

[0244] An audit report is generated based on multiple environmental factor values ​​corresponding to the audit event identifier.

[0245] For the description of the features in the embodiment corresponding to the material management device 200, please refer to the relevant description of the embodiment corresponding to the material management method, and will not be repeated here.

[0246] The material management device provided by the embodiments of the present disclosure improves the material recognition rate in various complex scenes such as dark light, shielding, reflection, and packaging, through complementary perception of millimeter wave radar (penetration characteristic) and visual sensor (high resolution characteristic). In addition, the nonlinear weight mapping function dynamically adjusts the millimeter wave radar weight and the visual sensor weight according to real-time environmental parameters (light intensity, texture complexity, outer packaging thickness, etc.). For example, in a dark light environment, the visual sensor weight is reduced, and the millimeter wave radar weight is increased, and vice versa. This dynamic adjustment mechanism ensures that the system can always use the most suitable sensor data for material recognition in a complex and variable environment, thereby significantly improving the recognition robustness. Through time synchronization processing, the consistency of radar point cloud data and image feature data in time is ensured, and fusion errors caused by time difference are avoided. Through spatial alignment processing, the consistency of data in spatial coordinates is ensured, so that the radar point cloud and the image feature can be accurately aligned in a unified coordinate system, thereby reducing the data correction and compensation required in the subsequent processing steps and improving the overall processing efficiency. Through fusion confidence, the borrowing and returning strategy of the target material can be determined, the occurrence of misoperation is avoided, the accuracy and efficiency of material management are improved, and the high-frequency borrowing and returning demand of materials in different environments is met.

[0247] The embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above material management method embodiments.

[0248] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above material management method embodiments when running.

[0249] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0250] The embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above material management method embodiments.

[0251] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned material management method embodiments.

[0252] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0253] The above is a detailed introduction to a material management method, device, equipment and medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A material management method, characterized in that: include: In response to the wake-up interrupt signal, acquiring radar perception data and visual perception data of the target material; The radar perception data includes radar point cloud data, and the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data; the visual perception data includes image feature data, and regional illumination intensity, texture complexity, and contour clarity extracted from the image feature data; Calculating a weight of the radar perception data using a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity, and calculating a weight of the visual perception data using a nonlinear weight mapping function according to the contour clarity, the motion speed, and the acceleration; Performing time synchronization processing and space alignment processing on the radar point cloud data and the image feature data to generate time-space aligned fusion data; Calculating a fusion confidence based on the fusion data, the environmental uncertainty factor, and the motion uncertainty factor, wherein the fusion confidence is used to indicate the credibility of the perception data of the target material; A borrowing and returning strategy corresponding to the target material is determined according to the fusion confidence.

2. The material management method according to claim 1, characterized in that: The determining, according to the fusion confidence, a borrowing and returning strategy corresponding to the target material includes: When the fusion confidence falls within a first confidence threshold interval, determining the borrowing and returning strategy to be controlling the electric lock to release the target material; When the fusion confidence falls within the second confidence threshold range, determining the borrowing and returning strategy to trigger the millimeter-wave radar and the visual sensor to independently identify the target material and perform a logical and cross-check; When the fusion confidence falls within a third confidence threshold range, determining that the borrowing and returning strategy is to generate a visual operation guide; Among them, the first confidence threshold interval, the second confidence threshold interval and the third confidence threshold interval do not overlap, the center value of the first confidence threshold interval is greater than the center value of the second confidence threshold interval, and the center value of the second confidence threshold interval is greater than the center value of the third confidence threshold interval.

3. The material management method according to claim 1, characterized in that: Calculating the weight of the radar perception data using a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity, and calculating the weight of the visual perception data using a nonlinear weight mapping function according to the contour clarity, the motion speed, and the acceleration, include: Calculating an initial weight of the radar perception data by a nonlinear weight mapping function according to the physical thickness of the outer packaging, the light intensity, and the texture complexity; Calculating an initial weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function; performing exponential normalization processing on the initial weight of the radar perception data to calculate the weight of the radar perception data; The weight of the visual perception data is calculated according to the weight of the radar perception data.

4. The material management method according to claim 3, characterized in that: Before calculating the weight of the radar perception data using a nonlinear weight mapping function according to the physical thickness of the outer package, the light intensity, and the texture complexity, and calculating the weight of the visual perception data using a nonlinear weight mapping function according to the contour clarity, the motion speed, and the acceleration, the method further includes: The physical thickness of the outer packaging, the movement speed, the acceleration, the light intensity, the texture complexity and the contour clarity are respectively normalized to obtain the normalized physical thickness of the outer packaging, the normalized movement speed, the normalized acceleration, the normalized light intensity, the normalized texture complexity and the normalized contour clarity.

5. The material management method according to claim 4, characterized in that: The calculating the initial weight of the radar perception data by a nonlinear weight mapping function according to the physical thickness of the outer package, the light intensity, and the texture complexity includes: The initial weight of the radar perception data is calculated by a nonlinear weight mapping function based on the normalized physical thickness of the outer packaging, the normalized light intensity, the normalized texture complexity, the first learning coefficient, and the second learning coefficient; wherein the first learning coefficient is a weight coefficient that balances the light intensity and the physical thickness of the outer packaging; and the second learning coefficient is a weight coefficient for texture complexity.

6. The material management method according to claim 4, characterized in that: The calculating the initial weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function includes: Calculating an adaptive speed attenuation coefficient of the target material according to the normalized motion speed and the normalized acceleration; According to the normalized contour clarity and the adaptive speed attenuation coefficient, the third learning coefficient and the fourth learning coefficient, the initial weight of the visual perception data is calculated by a nonlinear weight mapping function; the third learning coefficient is the contour clarity weight coefficient, and the fourth learning coefficient is the motion stability weight coefficient.

7. The material management method according to claim 1, characterized in that: The performing time synchronization processing and space alignment processing on the radar point cloud data and the image feature data to generate time-space aligned fusion data includes: At each sampling moment, the median of the Precision Time Protocol clock source, the Network Time Protocol clock source, and the Global Positioning System clock source is used as the reference time; Calculating the variance of the reference time within a preset sliding window, and calculating a clock jitter compensation value according to the variance of the reference time; Calculate a corrected synchronization timestamp based on the reference time and the clock jitter compensation value, and correct the first timestamp of the millimeter-wave radar and the second timestamp of the visual sensor based on the synchronization timestamp so that the synchronization error between the first timestamp and the second timestamp remains within a preset time length; Acquire an initial extrinsic parameter matrix of the millimeter-wave radar and the visual sensor, wherein the initial extrinsic parameter matrix includes an initial rotation parameter and an initial translation parameter; An objective function is constructed based on a weighted sum of a reprojection error and a motion consistency error; the reprojection error is used to represent the square of the distance between the radar point cloud data and the corresponding pixel point after being projected onto the image plane by the initial extrinsic parameter matrix; the motion consistency error is used to represent the square of the directional deviation between the two-dimensional motion vector calculated by the image optical flow and the three-dimensional motion vector measured by the radar Doppler after being projected by the initial extrinsic parameter matrix; Iterate the initial extrinsic parameter matrix using a preset algorithm until the reprojection error is less than a preset pixel value, thereby obtaining target rotation parameters and target translation parameters; The radar point cloud data and the image feature data are mapped to a preset spatial coordinate system according to the target rotation parameter and the target translation parameter to generate fused data after time and space alignment.

8. The material management method according to claim 7, characterized in that: The calculating the variance of the reference time within a preset sliding window, and calculating the clock jitter compensation value according to the variance of the reference time, includes: Calculating a sliding mean of the reference time within the preset sliding window; Calculating a clock jitter variance according to the reference time and a sliding mean of the reference time; Calculating an adaptive smoothing factor based on the precision time protocol clock source and the network time protocol clock source; A clock jitter compensation value is calculated according to the clock jitter variance and the adaptive smoothing factor.

9. The material management method according to claim 7, characterized in that: The objective function is constructed based on the weighted sum of the reprojection error and the motion consistency error, including: The objective function is constructed according to the following formula: ； in, represents the coordinates of the image feature points, represents the projection function that maps three-dimensional coordinates to two-dimensional coordinates, represents the camera intrinsic parameter matrix, represents the rotation parameter, represents the radar point cloud coordinates, represents the translation parameter, represents the image gradient field, represents the optical flow vector, represents the radar point motion vector, represents a balance factor, which is used to adjust the weights of the reprojection error term and the motion consistency error term.

10. The material management method according to claim 1, characterized in that: After determining the borrowing and returning strategy of the target material according to the fusion confidence, the method further includes: Taking a first preset time period as a period, obtaining multiple environmental factor values ​​of the current period; Normalizing the multiple environmental factor values, and concatenating the normalized multiple environmental factor values ​​to obtain a concatenated environmental parameter; Performing a hash operation on the splicing environment parameters to generate an environment fingerprint; Acquire multiple environmental fingerprints corresponding to multiple consecutive periods, and generate an environmental fingerprint chain based on the multiple environmental fingerprints; When a material borrowing or returning event is triggered, the environmental fingerprint chain is written into the blockchain.

11. The material management method according to claim 10, characterized in that: The method further comprises: In response to the audit event identifier input by the user, obtaining a target environment fingerprint chain corresponding to the audit event identifier; Perform reverse calculation based on the target environment fingerprint chain to obtain multiple environmental factor values ​​corresponding to the audit event identifier; An audit report is generated based on multiple environmental factor values ​​corresponding to the audit event identifier.

12. A material management device, characterized in that: include: A data acquisition module, configured to acquire radar perception data and visual perception data of a target material in response to a wake-up interrupt signal; The radar perception data includes radar point cloud data, and the physical thickness, motion speed, and acceleration of the outer packaging extracted from the radar point cloud data; the visual perception data includes image feature data, and regional illumination intensity, texture complexity, and contour clarity extracted from the image feature data; a weight calculation module, configured to calculate the weight of the radar perception data according to the physical thickness of the outer packaging, the light intensity, and the texture complexity through a nonlinear weight mapping function, and to calculate the weight of the visual perception data according to the contour clarity, the motion speed, and the acceleration through a nonlinear weight mapping function; a synchronization processing module, configured to perform time synchronization processing and spatial alignment processing on the radar point cloud data and the image feature data to generate fused data after time and space alignment; a confidence calculation module, configured to calculate a fusion confidence based on the fusion data, an environmental uncertainty factor, and a motion uncertainty factor, wherein the fusion confidence is used to represent the credibility of the perception data of the target material; A strategy determination module is used to determine the borrowing and returning strategy corresponding to the target material according to the fusion confidence.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the material management method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the material management method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the material management method according to any one of claims 1 to 11 are implemented.

Images