Warehouse handling equipment operation identification method and device, and computer device

Abstract

The application relates to a warehouse loading and unloading handling equipment operation identification method and device and computer equipment. The method comprises the following steps: obtaining a to-be-enhanced data set in a region with a collision risk and a to-be-optimized data set in a region without a collision risk from target point cloud data collected by loading and unloading handling equipment in a warehouse; performing obstacle detection on the to-be-enhanced data set to obtain an obstacle detection result; performing lightweight processing on the to-be-optimized data set to obtain an optimized data set; inputting the obstacle detection result and the optimized data set into a collision avoidance model to obtain a predicted running track of the loading and unloading handling equipment in a target period; performing operation identification on the loading and unloading handling equipment according to the predicted running track, and controlling the running of the loading and unloading handling equipment according to the identification result of the operation identification. The method can improve the accuracy of the identification result of the operation identification on the loading and unloading handling equipment.

Description

Technical Field

[0001] This application relates to the field of logistics and warehousing technology, and in particular to a method and apparatus for identifying the operation of loading and unloading equipment in a warehouse, as well as computer equipment. Background Technology

[0002] In the logistics and warehousing sector, efficient warehouse operation is crucial to a company's competitiveness. As the core execution unit of warehouse operations, the safety of loading and unloading equipment is of paramount importance.

[0003] In related technologies, millimeter-wave radar is installed on loading and unloading equipment. Based on the point cloud data collected by the millimeter-wave radar, the loading and unloading equipment is identified to obtain the identification result. The identification result indicates whether the loading and unloading equipment is about to encounter an obstacle. Then, the operation of the loading and unloading equipment is controlled according to the identification result to avoid collisions.

[0004] However, there is a problem with the inaccurate identification results of loading and unloading equipment in the relevant technologies. Summary of the Invention

[0005] Based on this, this application provides a method and apparatus for identifying the operation of loading and unloading equipment in a warehouse, as well as a computer device, which can improve the accuracy of the identification results for identifying the operation of loading and unloading equipment.

[0006] In a first aspect, this application provides a method for identifying the operation of loading and unloading equipment in a warehouse, the method comprising:

[0007] From the target point cloud dataset collected by loading and unloading equipment in the warehouse, obtain the dataset to be enhanced in areas with collision risk and the dataset to be optimized in areas without collision risk;

[0008] Obstacle detection is performed on the dataset to be enhanced to obtain obstacle detection results;

[0009] The dataset to be optimized is lightweighted to obtain the optimized dataset.

[0010] Both the obstacle detection results and the optimized dataset are input into the collision avoidance model to obtain the predicted operating trajectory of the loading and unloading equipment during the target time period;

[0011] The loading and unloading equipment is identified based on the predicted operating trajectory, and its operation is controlled based on the identification results.

[0012] In some embodiments, from the target point cloud dataset collected by loading and unloading equipment in the warehouse, a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk are obtained, including:

[0013] Obtain initial point cloud datasets from multiple target areas collected by loading and unloading equipment; where multiple target areas are regions within multiple different azimuth angle ranges collected by the loading and unloading equipment respectively, and the union of multiple different azimuth angle ranges is the omnidirectional angle;

[0014] Identify the point cloud data in each region with collision risk in the initial point cloud dataset of each target region, and stitch together the point cloud data in each region with collision risk to obtain the dataset to be enhanced.

[0015] The point cloud data outside the point cloud data in each of the regions with collision risks in the initial point cloud dataset of each target region are identified as the dataset to be optimized.

[0016] In some embodiments, obstacle detection is performed on the dataset to be enhanced to obtain obstacle detection results, including:

[0017] The farthest point sampling algorithm is used to sample a preset number of point cloud data from the dataset to be enhanced, resulting in sampled point cloud data.

[0018] Based on the sampled point cloud data, determine the updated obstacle detection model;

[0019] The updated obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the detection results are obtained.

[0020] In some embodiments, determining an updated obstacle detection model based on sampled point cloud data includes:

[0021] The ratio of the number of obstacle point cloud data to the total number of sampled point cloud data is determined as the obstacle proportion of the sampled point cloud data.

[0022] Based on the proportion of obstacles in the sampled point cloud data, determine the target weight of each obstacle category under multiple obstacle categories;

[0023] The preset obstacle detection model is updated based on the target weights of each obstacle category to obtain the updated obstacle detection model.

[0024] In some embodiments, the target weight of each obstacle category under multiple obstacle categories is determined based on the obstacle proportion of the sampled point cloud data, including:

[0025] Based on the proportion of obstacles in the sampled point cloud data, determine the smoothing coefficient used to smooth the labels;

[0026] The true labels of each obstacle category are smoothed using a smoothing coefficient to obtain the target labels for each obstacle category;

[0027] A pre-defined obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the probability of the dataset to be enhanced in each obstacle category is obtained.

[0028] Based on the target label and probability of each obstacle category, the loss value of each obstacle category is determined, and the target weight of each obstacle category is determined based on the loss value of each obstacle category.

[0029] In some embodiments, the dataset to be optimized is subjected to lightweight processing to obtain an optimized dataset, including:

[0030] The point cloud data in the dataset to be optimized is segmented to obtain multiple sub-datasets in the segmented regions;

[0031] Identify at least one subset of data from a non-edge region among subsets of data from multiple segmented regions;

[0032] The subsets in each non-edge region are replaced with the centroid coordinates of each non-edge region to obtain the optimized dataset.

[0033] In some embodiments, identifying at least one subset of data from a non-edge region from subsets of data from multiple segmented regions includes:

[0034] Determine the normal vector between each target point and multiple neighboring points in a subset of data within each segmented region;

[0035] Based on the normal vector, determine the average angle of the subsets in each segmentation region;

[0036] The subsets of data in each segmented region corresponding to the average angle that is less than or equal to the preset angle are determined as subsets of data in each non-edge region.

[0037] In some embodiments, controlling the operation of loading and unloading equipment based on the identification results of the job identification includes:

[0038] If the identification result of the operation indicates that the probability of a collision occurring within the target time period is greater than or equal to the first threshold and less than the second threshold, the loading and unloading equipment is controlled to issue an audible alarm at the first alarm frequency.

[0039] If the identification result of the operation indicates that the probability of a collision occurring within the target time period is greater than or equal to the second threshold, the loading and unloading equipment is controlled to issue an audible alarm at the second alarm frequency, and the control resistance of the loading and unloading equipment is increased; the second alarm frequency is greater than the first alarm frequency.

[0040] Secondly, this application provides an operation identification device for loading and unloading equipment in a warehouse, the device comprising:

[0041] The data acquisition module is used to acquire, from the target point cloud dataset collected by loading and unloading equipment in the warehouse, the dataset to be enhanced in areas with collision risk and the dataset to be optimized in areas without collision risk.

[0042] The obstacle detection module is used to detect obstacles in the dataset to be enhanced and obtain the obstacle detection results.

[0043] The lightweight processing module is used to perform lightweight processing on the dataset to be optimized, resulting in an optimized dataset.

[0044] The trajectory prediction module is used to input both obstacle detection results and optimized datasets into the collision avoidance model to obtain the predicted trajectory of the loading and unloading equipment during the target time period.

[0045] The identification and control module is used to identify the operation of the loading and unloading equipment based on the predicted operating trajectory, and to control the operation of the loading and unloading equipment based on the identification results.

[0046] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any one of the first aspects.

[0047] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method of any one of the first aspects.

[0048] Fifthly, this application provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method of any one of the first aspects.

[0049] In the technical solution provided in this application embodiment, the target point cloud dataset is divided into a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk. Obstacle detection is performed on the dataset to be enhanced in areas with collision risk, focusing on areas where collisions may occur. This allows for more concentrated detection of these potentially colliding areas, improving the accuracy of obstacle detection and reducing the impact of information unrelated to collision risk on the detection results. This improves the accuracy of the determined obstacle detection results, thereby improving the accuracy of the identification results for the loading and unloading equipment. Furthermore, by performing lightweight processing on areas without collision potential, the computational load is reduced, improving the real-time performance and efficiency of the operation identification method. This improves the timeliness of the identified results for the operation identification of the loading and unloading equipment, avoiding the problem of reduced accuracy due to delays in collision risk determination, and thus improving the accuracy of the identification results for the operation identification of the loading and unloading equipment. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a method for identifying the operation of loading and unloading equipment in a warehouse, provided in some embodiments;

[0052] Figure 2 A flowchart illustrating a method for obtaining a dataset to be enhanced and a dataset to be optimized, provided in some embodiments;

[0053] Figure 3 A flowchart illustrating a method for obtaining obstacle detection results provided in some embodiments;

[0054] Figure 4 A flowchart illustrating a method for obtaining an optimized dataset by lightweighting a dataset to be optimized, as provided in some embodiments;

[0055] Figure 5 A flowchart illustrating a method for operation identification and control of loading and unloading handling equipment provided in some embodiments;

[0056] Figure 6 A schematic diagram of the operation identification device for loading and unloading equipment in a warehouse provided in some embodiments;

[0057] Figure 7 A schematic diagram of the structure of a computer device provided for some embodiments. Detailed Implementation

[0058] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0060] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined. In the description of the embodiments of this application, "each" means each of the multiple options, unless otherwise explicitly defined.

[0061] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0062] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0063] The computer equipment in this application embodiment may include one or a combination of at least two of the following: loading and unloading equipment, server, computer, industrial control computer, etc. For example, the loading and unloading equipment may include loading and unloading vehicles or robots, etc. For example, the loading and unloading vehicles may include forklifts, pallet trucks, tractors, automated guided vehicles, or trailers, etc.

[0064] Figure 1 A flowchart illustrating a method for identifying the operation of loading and unloading equipment in a warehouse, as provided in some embodiments, is shown below. Figure 1 As shown, this method is applied to a computer device, and the method includes:

[0065] S101. From the target point cloud dataset collected by the loading and unloading equipment in the warehouse, obtain the dataset to be strengthened in the area with collision risk and the dataset to be optimized in the area without collision risk.

[0066] For example, loading and unloading equipment can operate within a warehouse. For example, the loading and unloading equipment is equipped with millimeter-wave radar, which collects a target point cloud dataset within the warehouse. For example, the warehouse contains obstacles such as shelves, goods, and building structures (e.g., columns).

[0067] Each point cloud data point represents a three-dimensional coordinate system. In some embodiments, the target point cloud dataset can be a point cloud dataset covering all angles detected by the loading and unloading equipment. Here, the all-around angle refers to angles within a horizontal 360° range. In other embodiments, to reduce computational load, the target point cloud dataset can be a point cloud dataset covering the front and sides detected by the loading and unloading equipment.

[0068] For example, the target point cloud dataset can be point cloud data collected within a specified time period. Alternatively, the target point cloud dataset can be point cloud data collected at a single point in time. Here, the specified time period can be the period between a specified time and the current time; the specified time is a time preceding the current time and spaced at a specified interval from the current time.

[0069] For example, areas with collision risk may include areas where there is a possibility of collision with buildings, goods, or shelves during the target time period. For example, areas without collision risk may include areas where there is no possibility of collision with buildings, goods, or shelves during the target time period. These areas without collision risk can be the areas within the detection area of ​​the loading and unloading equipment excluding areas with collision risk.

[0070] For example, areas where collisions are unlikely can include open areas where the distance between the loading / unloading equipment and fixed structures is relatively large (straight-line distance > safety threshold). Fixed structures include, for example, dynamic obstacles (such as moving goods, other vehicles besides the loading / unloading equipment) and static obstacles (such as shelves, fixed goods, and buildings). In areas where collisions are likely, areas where the distance between the loading / unloading equipment and fixed structures is relatively small (straight-line distance ≤ safety threshold) are considered to indicate a potential collision. Here, the actual direction of travel of the loading / unloading equipment is not considered; only whether it is within the designated area is taken into account.

[0071] The dataset to be enhanced and the dataset to be optimized can be obtained by partitioning the target point cloud dataset. In some embodiments, point cloud data with a density greater than or equal to a preset density are identified as the dataset to be enhanced in areas with collision risk, while data outside the dataset to be enhanced are identified as the dataset to be optimized. In other embodiments, classification models such as support vector machines, random forests, and neural networks (such as PointNet, a deep learning network for point cloud data) are used to divide the target point cloud dataset into the dataset to be enhanced in areas with collision risk and the dataset to be optimized in areas without collision risk.

[0072] S102. Perform obstacle detection on the dataset to be enhanced to obtain obstacle detection results.

[0073] For example, the updated obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the detection results of the obstacles are obtained.

[0074] In some embodiments, obstacle detection results may include the type of obstacle (e.g., a building, shelf, or goods). In other embodiments, obstacle detection results may include the type of obstacle and point cloud data of the area where the obstacle is located in an area with collision risk. Exemplarily, an obstacle detection model is used to identify obstacle point cloud data in point cloud data, as well as the category of the obstacle.

[0075] For example, the updated obstacle detection model can be a preset obstacle detection model, or it can be a detection model obtained by updating the preset obstacle detection model according to the target weights of each obstacle category. For example, the obstacle detection model can include a YOLO model. The target weights of each obstacle category are a set of parameters in the YOLO model used to measure the importance of different obstacle types in the prediction.

[0076] In one implementation, historical point cloud data, historical obstacle point cloud data, and historical attention weights are acquired, and these data are input into a preset model for training. The output of the preset model is used as the target model parameter, and the preset model is updated according to the target model parameter to obtain the obstacle detection model.

[0077] In one implementation, historical point cloud data includes a wide variety of point cloud data, such as point cloud data of main objects in a warehouse, including pillars, shelves, goods of various sizes, and other obstacles, as well as point cloud data of the surrounding environment. Historical obstacle point cloud data (containing only the main objects and excluding the surrounding environment) is also included. Historical attention weights are used to force the model to focus on key regions (obstacles that are difficult to distinguish).

[0078] S103. Perform lightweight processing on the dataset to be optimized to obtain the optimized dataset.

[0079] For example, lightweight processing may include at least one of the following: uniform sampling, random sampling, filtering, feature point extraction, and centroid coordinate data replacement.

[0080] In some embodiments, performing lightweight processing on the dataset to be optimized to obtain an optimized dataset may include: lightweighting the non-edge regions of the dataset to be optimized to obtain an optimized dataset. For example, replacing point cloud data in the non-edge regions of the dataset to be optimized with centroid coordinate data to obtain an optimized dataset.

[0081] In other embodiments, the dataset to be optimized is subjected to lightweight processing to obtain an optimized dataset. This may include performing at least one of the following processes on the dataset to be optimized: uniform sampling, random sampling, filtering, feature point extraction, etc., to obtain the optimized dataset. For example, uniform sampling is performed on the dataset to be optimized to obtain the optimized dataset.

[0082] S104. Input both the obstacle detection results and the optimized dataset into the collision avoidance model to obtain the predicted operating trajectory of the loading and unloading equipment during the target time period.

[0083] The collision avoidance model takes obstacle detection results and an optimized dataset as input, and outputs the predicted trajectory of the loading and unloading equipment during the target time period. For example, the obstacle detection results and optimized dataset can be: obstacle detection results and optimized datasets at multiple time points corresponding to point cloud data collected within a specified time period, respectively. Alternatively, the obstacle detection results and optimized dataset can be: obstacle detection results and optimized datasets corresponding to point cloud data collected at the current time, respectively.

[0084] The target time period is a time interval with a preset duration starting from the current time.

[0085] For example, the collision avoidance model is trained based on time-series point cloud data and has trajectory prediction capabilities. For example, the warehouse environment can be scanned and collected at different time points, and the collected point cloud data can be arranged in chronological order to form time-series data. Areas with collision risks can be identified from the time-series data, and obstacle detection results for these areas can be obtained through obstacle detection. A lightweight processing method is applied to the point cloud data of areas without collision risks in the time-series data to obtain a dataset. Based on the obstacle detection results of areas with collision risks in the time-series data and the lightweight processing dataset, a collision avoidance model is generated.

[0086] In some embodiments, different collision avoidance detection models can be selected based on actual needs and computational resources. For example, if high accuracy and real-time performance are required, deep learning-based networks (such as RadarMFNet) can be used. Alternatively, if computational resources are limited, spatiotemporal convolutional networks can be used.

[0087] S105. Based on the predicted operating trajectory, identify the operation of the loading and unloading equipment, and control the operation of the loading and unloading equipment based on the identification results.

[0088] In some embodiments, the operation identification of the loading and unloading equipment based on the predicted operating trajectory includes: determining the distance between the loading and unloading equipment and the nearest obstacle within a target time period based on the predicted operating trajectory; determining the identification result of the operation identification of the loading and unloading equipment based on the distance between the loading and unloading equipment and the nearest obstacle within the target time period; the identification result represents the probability of a collision occurring within the target time period.

[0089] In some embodiments, if a collision risk is determined based on the identification results of the operation identification, the driver or manager can be alerted and a risk warning can be issued.

[0090] In the technical solution provided in this application embodiment, the target point cloud dataset is divided into a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk. Obstacle detection is performed on the dataset to be enhanced in areas with collision risk, focusing on areas where collisions may occur. This allows for more concentrated detection of these potentially colliding areas, improving the accuracy of obstacle detection and reducing the impact of information unrelated to collision risk on the detection results. This improves the accuracy of the determined obstacle detection results, thereby improving the accuracy of the identification results for the loading and unloading equipment. Furthermore, by performing lightweight processing on areas without collision potential, the computational load is reduced, improving the real-time performance and efficiency of the operation identification method. This improves the timeliness of the identified results for the operation identification of the loading and unloading equipment, avoiding the problem of reduced accuracy due to delays in collision risk determination, and thus improving the accuracy of the identification results for the operation identification of the loading and unloading equipment.

[0091] In addition, controlling the operation of loading and unloading equipment through more accurate operation identification results can improve the reliability of control, enhance the anti-collision capability of loading and unloading equipment in complex warehouse environments, and improve the operational safety of loading and unloading equipment.

[0092] In the technical solution provided in this application embodiment, by dividing the acquired target area point cloud data into a dataset to be enhanced and a dataset to be optimized, enhancing the dataset to be enhanced, and lightweighting the dataset to be optimized, and combining it with an anti-collision model for prediction and operational safety identification, effective monitoring and risk warning of the operational safety of loading and unloading equipment is achieved.

[0093] The technical solution provided in this application can predict changes in the distance between loading and unloading equipment and obstacles in advance and issue early warning signals in a timely manner, thereby effectively ensuring the safety of loading and unloading equipment operations. Furthermore, by enhancing the dataset to be enhanced and lightweighting the dataset to be optimized, the accuracy of the determined obstacle detection results can be improved, the amount of data input to the collision avoidance model can be reduced, the speed of determining the predicted trajectory of the loading and unloading equipment in the target time period can be increased, and the computational resources for the collision avoidance model can be saved. Therefore, by rationally allocating computational resources, the performance of computer equipment is optimized, and the collision avoidance capability of loading and unloading equipment in complex warehouse environments is improved.

[0094] Figure 2 A flowchart illustrating the methods for obtaining the dataset to be enhanced and the dataset to be optimized, as provided in some embodiments, is shown below. Figure 2 As shown, Figure 2 The embodiment describes step S101, and the method is applied to a computer device. The method includes:

[0095] S1011. Obtain the initial point cloud dataset of multiple target areas collected by the loading and unloading equipment; wherein, the multiple target areas are areas within multiple different azimuth angle ranges collected by the loading and unloading equipment respectively, and the union of multiple different azimuth angle ranges is the omnidirectional angle.

[0096] In some embodiments, the loading and unloading equipment may be equipped with multiple millimeter-wave radars, each with a detection area of ​​a target area, so that the multiple millimeter-wave radars respectively detect initial point cloud datasets in multiple target areas. Exemplarily, overlapping areas may exist among the multiple target areas. Exemplarily, the target area is the data area acquired by the millimeter-wave radars installed on the operating vehicle; for example, if the loading and unloading equipment is equipped with five millimeter-wave radars, then there are five target areas.

[0097] For example, multiple target areas may include at least two of the following: directly in front of, to the left of, to the right of, to the left front, to the right front, to the left rear, to the right rear, and directly behind the loading and unloading equipment.

[0098] Each angle range can be one angle range within a horizontal 360° range. For example, adjacent angle ranges may overlap. For instance, taking the area directly in front of the loading / unloading equipment as 0°, the different angle ranges could be 315° to 360°, 0° to 45°, 30° to 120°, 90° to 180°, 180° to 270°, 240° to 330°, etc.

[0099] By combining multiple different azimuth angle ranges into a full-range angle, it is possible to collect point cloud data of loading and unloading equipment within a horizontal 360° range.

[0100] S1012. Identify the point cloud data in each region with collision risk in the initial point cloud dataset of each target region, and stitch together the point cloud data in each region with collision risk to obtain the dataset to be enhanced.

[0101] For each millimeter-wave radar, an analysis of the areas with collision risk is performed on the initial point cloud dataset of a target area, resulting in point cloud data of the areas with collision risk in the initial point cloud dataset of each target area.

[0102] Since two adjacent azimuth angle ranges overlap in multiple different azimuth angle ranges, the point cloud data in the area with collision risk are stitched together (or merged). That is, the point cloud data of the same area (or the same azimuth angle range) in the area with collision risk in the point cloud data collected by different millimeter-wave radars are taken as the union set to obtain the dataset to be enhanced. For example, taking multiple different azimuth angle ranges, including the range between 30° and 120° and the range between 90° and 180°, as an example, the range between 30° and 120° overlaps with the range between 90° and 120°. Therefore, the point cloud data in the area with collision risk within the range between 90° and 120° within the range between 30° and 120° can be combined with the point cloud data in the area with collision risk within the range between 90° and 120° within the range between 90° and 180°. The point cloud data in the area with collision risk within the range between 30° and 90°, the point cloud data after combining the points cloud data in the area with collision risk within the range between 90° and 120°, and the point cloud data in the area with collision risk within the range between 120° and 180° can be determined as the dataset to be enhanced.

[0103] The sparse point cloud acquired by millimeter-wave radar (e.g., missing data from the top of shelves or the edges of goods) leads to incomplete 3D modeling and an inability to accurately identify obstacles in blind spots. In some possible implementation scenarios, the X, Y, and Z axes of the warehouse are set. The radar acquires local point clouds P1 (top of a 3m high shelf) of the front shelf, P2 (top of goods transported by loading and unloading equipment), and P3 (boxes 1m high) of goods on the side. P1, P2, and P3 are then stitched together to obtain P (P∈R). 3M ×3To augment the dataset, point clouds from different perspectives are supplemented to simulate a complete 3D environment (e.g., loading and unloading equipment carrying goods, with shelves and goods coexisting around the equipment). P represents the dataset to be augmented. The furthest point sampling algorithm is used on P to sample the furthest points, retaining 1024 points (i.e., sampled point cloud data). Points at the top of the shelves and the edges of the goods (key contours) are prioritized to generate new samples. For example, the number of point clouds from the shelf is m1=600, and the number of points from the goods is m2=424.

[0104] In this approach, by stitching together local point cloud data from different perspectives (such as point clouds of the top of shelves, the top of goods, and the side goods), a more complete three-dimensional environment can be simulated. This stitching method effectively supplements the sparse point cloud data collected by millimeter-wave radar, reducing blind spots caused by radar coverage or angle limitations, thereby providing richer environmental information for subsequent obstacle detection.

[0105] S1013. The point cloud data outside the point cloud data in each region with collision risk in the initial point cloud dataset of each target region is determined as the dataset to be optimized.

[0106] In some embodiments, the dataset to be optimized may be point cloud data excluding the point cloud data in the collision-risk areas of the initial point cloud datasets in each target region that are not stitched together. In other embodiments, the point cloud data excluding the point cloud data in the collision-risk areas of the initial point cloud datasets in each target region may be stitched together within the same azimuth angle range to obtain the dataset to be optimized.

[0107] In this embodiment, the datasets in the dataset to be enhanced are aggregated according to the region type (i.e., multiple different azimuth angle ranges, for example, when loading and unloading equipment is moving in the warehouse, the radar collects local point cloud data of the front shelf and local point cloud data of the goods in the side front respectively. The two shelves are the same, but the collection perspectives are different) to obtain an aggregated dataset (i.e., the stitched dataset). By aggregating according to the region type, point clouds from different perspectives are supplemented to simulate a complete environment. This avoids the situation where the point cloud data collected by millimeter-wave radar may be incomplete or insufficient when the loading and unloading equipment moves in the warehouse. Thus, by supplementing the point cloud data from different perspectives, the diversity of point cloud data can be increased, making up for the deficiencies of single collection data, thereby providing richer environmental information for 3D modeling.

[0108] The system enhances a portion of the point cloud data (i.e., enhances the dataset to be enhanced by stitching together point clouds) while lightweighting another portion. This is based on the need to optimize collision risks and computing resources in different regions. This achieves two objectives: First, it enables the rational allocation of computing resources, concentrating limited resources in high-risk areas while reducing the overall computational burden through lightweighting. Second, through enhancement and lightweighting, the system can improve real-time performance and efficiency while maintaining collision detection accuracy, thereby better ensuring the safety of loading and unloading equipment operations.

[0109] In the technical solution provided in this application embodiment, point cloud data in each region with collision risk in the initial point cloud dataset of each target region is identified, and the point cloud data in each region with collision risk is stitched together to obtain a dataset to be enhanced. This can improve the diversity of point cloud data in the dataset to be enhanced, make up for the insufficiency of single data acquisition, provide richer environmental information for 3D modeling, and improve the accuracy of obstacle detection results by increasing the richness of the dataset to be enhanced.

[0110] Figure 3 A flowchart illustrating a method for obtaining obstacle detection results provided in some embodiments, such as... Figure 3 As shown, Figure 3 The embodiment describes step S102, and the method is applied to a computer device. The method includes:

[0111] S1021. The farthest point sampling algorithm is used to sample a preset number of point cloud data from the dataset to be enhanced, and the sampled point cloud data is obtained.

[0112] Farthest-point sampling is used to sample M points from a point cloud of N points, so that these points can represent the overall contour of the point cloud relatively well. The idea behind farthest-point sampling is to iteratively select M points from the N-point point cloud. Each selection is compared with the currently selected point set S={P0,P1,…,P…}. i Add the point with the largest minimum distance from all points in the set S to the set S.

[0113] The preset quantity is M, where M is an integer greater than or equal to 2. For example, the value of M can be determined based on the size of the obstacle. For example, if the obstacle's size is greater than the preset size, the value of M can be larger; if the obstacle's size is less than or equal to the preset size, the value of M can be smaller. For example, the quantity of M is directly proportional to the size of the obstacle.

[0114] In some embodiments, the dataset to be enhanced is clustered to obtain multiple clusters, and the farthest point sampling algorithm is used to sample a preset number of point cloud data for each cluster to obtain sub-sampled point cloud data of each cluster. The sub-sampled point cloud data of each cluster is determined as the sampled point cloud data.

[0115] S1022. Based on the sampled point cloud data, determine the updated obstacle detection model.

[0116] In some embodiments, determining an updated obstacle detection model based on sampled point cloud data includes: determining target weights for each obstacle category under multiple obstacle categories based on the sampled point cloud data; and updating a preset obstacle detection model based on the target weights for each obstacle category to obtain an updated obstacle detection model. For example, the target weights can be the weights in the loss function of the obstacle detection model. Specifically, updating the preset obstacle detection model based on the target weights for each obstacle category to obtain the updated obstacle detection model may include: replacing the weights in the loss function of the preset obstacle detection model with the target weights to obtain the updated obstacle detection model.

[0117] In other embodiments, determining an updated obstacle detection model based on sampled point cloud data includes: determining the ratio of the number of obstacle point cloud data to the total number of sampled point cloud data as the obstacle proportion of the sampled point cloud data; updating a preset obstacle detection model based on the obstacle proportion to obtain an updated obstacle detection model.

[0118] In some embodiments, the updated obstacle detection model is determined based on the sampled point cloud data, including: determining the obstacle proportion of the sampled point cloud data as the ratio of the number of obstacle point cloud data to the total number of sampled point cloud data; determining the target weight of each obstacle category under multiple obstacle categories based on the obstacle proportion of the sampled point cloud data; and updating the preset obstacle detection model according to the target weight of each obstacle category to obtain the updated obstacle detection model. In this way, considering that some obstacle categories may be more easily misclassified or missed in real-world scenarios, the updated obstacle detection model can assign higher weights to these difficult-to-classify obstacles through dynamic weight adjustment (i.e., dynamically adjusting the target weights), thereby reducing misclassification and missed detection. The dynamic adjustment mechanism can effectively improve the model's adaptability to imbalanced data, and when obstacles are easy to detect, their weights can be appropriately reduced to reduce the consumption of additional computing power.

[0119] For example, the obstacle proportion can be the sum of the proportions of each type of obstacle. For example, it can be determined whether each point cloud data in the sampled point cloud data is an obstacle (for example, obstacle point cloud data in the sampled point cloud data can be determined by an updated obstacle detection model), thereby determining the obstacle proportion. Furthermore, for example, it can be determined that the obstacle type corresponds to the sub-sampled point cloud data of each cluster, and the ratio between the number of clusters corresponding to each obstacle type and the total number of clusters is determined as the proportion of each type of obstacle; the sum of the proportions of each type of obstacle is then determined as the obstacle proportion.

[0120] In some embodiments, the target weight of each obstacle category under multiple obstacle categories is determined based on the proportion of obstacles in the sampled point cloud data, including: determining the proportion of each type of obstacle as the target weight of each obstacle category.

[0121] In other embodiments, the target weights of each obstacle category under multiple obstacle categories are determined based on the obstacle proportion of the sampled point cloud data. This includes: determining a smoothing coefficient for smoothing the labels based on the obstacle proportion of the sampled point cloud data; smoothing the real labels of each obstacle category (real labels include labels of real obstacles such as buildings, shelves, and goods) using the smoothing coefficient to obtain the target labels for each obstacle category; performing obstacle detection on the dataset to be strengthened using a preset obstacle detection model to obtain the probability of the dataset to be strengthened in each obstacle category; determining the loss value of each obstacle category based on the target labels and probabilities of each obstacle category; and determining the target weights of each obstacle category based on the loss values ​​of each obstacle category. The obstacle proportion can determine the model's recognition tendency, which refers to the model's inherent preference for certain obstacle categories, determined by the training data distribution and loss function design.

[0122] In some embodiments, determining the target weight of each obstacle category based on the loss value of each obstacle category may include: obtaining the total loss value of the loss values ​​of each obstacle category, and determining the ratio of the loss value of each obstacle category to the total loss value as the target weight of each obstacle category.

[0123] For example, if the proportion of obstacles is high, it indicates that there are many obstacles in the scene and the dataset contains rich information about obstacles. In this case, the smoothing coefficient can be appropriately reduced, allowing the model to focus more on the actual label information and fully utilize this rich data to learn the characteristics of obstacles for accurate identification and location. If the proportion of obstacles is low, it means that there is relatively little obstacle information in the data, and there may be noise or incompleteness. Increasing the smoothing coefficient can smooth the limited label information, reduce the impact of noise, and use the contextual information of the surrounding data to infer the possible location and attributes of obstacles, thereby improving the accuracy of obstacle recognition.

[0124] The following is one method for calculating the loss value: Treating the building, like a pillar, and the shelves as immovable, we can consider them as a single entity, treating both the building and the shelves as shelves. We set the actual labels for the shelves and goods to one-hot encoding (e.g., shelves as [1,0], goods as [0,1]). After label smoothing, the actual labels are adjusted as follows: ,in, K is the smoothing coefficient (0.4 is used as an example; for instance, the smoothing coefficient can be determined based on the proportion of obstacles in the sampled point cloud data), and K is the number of categories (set to 2).

[0125] Original shelf label c1=[1,0], after smoothing:

[0126] C1 smooth =0.6*[1,0]+0.4*[0.5,0.5]=[0.6*1+0.4*0.5,0.6*0+0.4*0.5]=[0.8,0.2].

[0127] Original cargo label c2=[0,1], after smoothing: c2 smooth =0.6*[0,1]+0.4[0.5,0.5]=[0.2,0.8].

[0128] The obstacle detection model (i.e., the preset obstacle detection model) predicts the following obstacles: Model output probability distribution f( Given a probability of 90% for the item to be on the shelf (0.9, 0.1), the cross-entropy loss formula is used. Substitute the values ​​into the calculation. , approximately equal to 0.1; model output probability distribution f( ))=[0.3,0.7] (meaning the probability of predicting it as a cargo is 70%), using the cross-entropy loss formula: Substitute the values ​​into the calculation. Approximately 0.3; shelf loss The model accurately predicted the shelf portion (probability 0.9, close to the true label's 0.8), resulting in lower losses; goods loss... The model performed poorly in predicting the cargo portion (probability 0.7 deviated from the true label by 0.8), resulting in a high loss.

[0129] In one implementation method, the formula for calculating the target weight is: shelf ,goods It can be seen that cargo classification is more difficult (higher loss), and should be given higher weight in the total loss, forcing the model to focus on error-prone areas. The point cloud density of goods at the top and bottom of the shelf is increased to reduce blind spot missed detections and improve the obstacle detection model's ability to focus on obstacles that are not easy to identify.

[0130] S1023. The updated obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the detection results of the obstacles are obtained.

[0131] In the technical solution provided in this application embodiment, sampling the farthest point of the stitched point cloud data can preserve key contour points (such as points on the top of the shelf and the edge of the goods) and generate new sample data (i.e., sampled point cloud data). This method can reduce the amount of data while prioritizing the retention of feature points that are crucial for obstacle recognition; selecting a preset number of representative points from a large amount of point cloud data reduces the amount of data for subsequent processing and lowers the computational complexity; prioritizing the selection of key contour points (such as points on the top of the shelf and the edge of the goods) can better reflect the geometric features and shape of obstacles, thereby improving the accuracy of obstacle recognition.

[0132] Figure 4 The following is a flowchart illustrating a method for obtaining an optimized dataset by lightweighting a dataset to be optimized, as provided in some embodiments. Figure 4 As shown, Figure 4 The embodiment describes step S103, and the method is applied to a computer device. The method includes:

[0133] S1031. Segment the point cloud data in the dataset to be optimized to obtain multiple sub-datasets in the segmented regions.

[0134] For example, the multiple segmented zones can be multiple different road surface areas within a road surface area where there is no collision risk. For example, the different road surfaces can be road surfaces in different directions and / or in different areas.

[0135] In some embodiments, the point cloud data in the dataset to be optimized can be segmented according to a grid division method.

[0136] In other embodiments, the point cloud data in the dataset to be optimized can be segmented into road surface regions according to different road surface attributes. For example, different road surface regions within a warehouse may have different heights, such as loading / unloading platforms and ordinary ground. By calculating the height value of each point in the point cloud and dividing the region based on a set height threshold, the point cloud data can be segmented. As another example, roads with different functions may have different slopes; for example, the driving lanes for loading and unloading equipment are horizontal, while drainage areas may have a certain slope. The point cloud data can be segmented by calculating the slope of local areas in the point cloud. As yet another example, points within the same road surface region are usually interconnected; the connectivity between points can be used to segment the point cloud data. As yet another example, the point cloud data in the dataset to be optimized can be input into a machine learning model (e.g., support vector machine or random forest) so that the machine learning model outputs multiple subsets of segmented regions. As yet another example, the point cloud data in the dataset to be optimized can be input into a deep learning-based segmentation model (e.g., PointNet, a deep learning model for processing point cloud data) so that the deep learning-based segmentation model outputs multiple subsets of segmented regions.

[0137] S1032. Identify at least one subset of data from a non-edge region among the subsets of data from multiple segmented regions.

[0138] For example, from Q segmented regions, identify P non-edge regions. P is an integer less than Q.

[0139] In some embodiments, identifying at least one subset of a non-edge region from subsets of multiple segmented regions includes: determining the normal vector between each target point and multiple neighboring points in the subset of each segmented region; determining the average angle of the subsets of each segmented region based on the normal vector; and identifying the subsets of each segmented region corresponding to the average angle less than or equal to a preset angle as subsets of each non-edge region.

[0140] S1033. Replace the subsets in each non-edge region with the centroid coordinates of each non-edge region to obtain the optimized dataset.

[0141] In the technical solution provided in this application embodiment, the point cloud data in the dataset to be optimized is segmented into multiple subsets (i.e., subsets in the multiple segmented regions mentioned above). The normal vectors of the target points and their neighboring points in each subset are determined, and the average angle in the subset is calculated based on the normal vectors. Each subset contains multiple sub-data points, and each sub-data point contains multiple target points (for example, each sub-data point is a target point; or, from the points corresponding to multiple sub-data points, a number of target points are randomly or uniformly selected as multiple target points). Each target point has multiple neighboring points. If the average angle is less than or equal to a preset angle, the region corresponding to the subset is determined to be a non-edge region. The centroid coordinates of the subset are calculated, and the coordinates in the subset are replaced based on the centroid coordinates to obtain the target sub-data. The optimized dataset is then determined based on each target sub-data.

[0142] In the technical solution provided in the embodiments of this application, the data in the dataset to be optimized does not have a significant effect on collision avoidance detection, so lightweight processing is required. However, lightweight processing is prone to over-processing, resulting in geometric size reduction or edge blurring. Therefore, a scheme for edge detection of point cloud data is proposed (that is, in the subsets of multiple segmented regions, the subsets outside the subsets of each non-edge region are determined as the subsets of edge regions).

[0143] The subset contains multiple sub-data sets, each corresponding to a region. Each subset contains multiple target points, and each target point has multiple neighboring points. The normal vector of each target point is calculated. Where Cov represents the covariance matrix, Let O be the coordinates of a neighboring point, O be the centroid (geometric center, i.e., the average value of all neighboring points), and k be the number of neighboring points. The vector represents the offset between the neighborhood point and the centroid, and T represents the transpose of the matrix. This represents the matrix multiplication of the offset vector and its transpose (to generate a 3×3 matrix). Eigenvalue decomposition is performed on the covariance matrix to obtain three eigenvalues ​​(λ1, λ2, λ3) and corresponding eigenvectors (v1, v2, v3). The eigenvector corresponding to the smallest eigenvalue is determined as the normal vector between the target point and each of its neighboring points. The normal vector of each target point can be obtained through this method.

[0144] In one implementation, the centroid coordinates of a subset of data are calculated as follows: , where O represents the average coordinates (centroid) of the neighborhood points. Representing the target point Let the coordinates of the j-th neighboring point be given, and k be the number of neighboring points. Calculate the average included angle of the target point: ,in, The formula above, representing the average angle (i.e., the average angle of the subset of data), means that for each target point... Calculate its normal vector Normal vectors of all neighboring points The included angle Finally, for all target points within the sub-data... Take the average to obtain the average angle of the sub-data. Because the normal vectors of edge points differ significantly from those of their neighboring points, while those of non-edge points tend to be similar, the average angle... If the angle is less than or equal to the preset angle, it is determined to be a non-edge area. By accurately identifying edge points, redundant data interference in flat areas is avoided.

[0145] In this embodiment, the point cloud data in the dataset to be optimized is segmented to obtain a subset of data. The normal vectors of the target point and each neighboring point in the subset of data are determined, and the average angle in the subset of data is calculated based on the normal vectors. The subset of data contains multiple subsets, each subset of data contains multiple target points, and each target point has multiple neighboring points. If the average angle is less than or equal to a preset angle, the region corresponding to the subset of data is determined to be a non-edge region. The centroid coordinates of the subset of data are calculated, and the coordinates in the subset of data are replaced based on the centroid coordinates to obtain the target subset of data. The optimized dataset is determined based on each target subset of data.

[0146] In the technical solution provided in this application embodiment, by determining the subset of data in each segmented region corresponding to the average included angle that is less than or equal to a preset included angle as the subset of data in each non-edge region, edge points can be accurately identified, and redundant data interference in flat regions can be avoided.

[0147] Figure 5 A flowchart illustrating a method for operation identification and control of loading and unloading equipment provided in some embodiments, such as... Figure 5 As shown, Figure 5 The embodiment describes S105, and the method is applied to a computer device. The method includes:

[0148] S1051. Based on the predicted operating trajectory, perform operation identification on the loading and unloading equipment to obtain the operation identification result.

[0149] In some embodiments, the shortest distance between the predicted running trajectory for the target time period and the obstacle can be obtained, and the identification result of the operation identification, which characterizes the probability of a collision occurring within the target time period, can be determined based on the shortest distance.

[0150] S1052. If the identification result of the operation identification indicates that the probability of a collision occurring within the target time period is greater than or equal to the first threshold and less than the second threshold, control the loading and unloading equipment to issue an audible alarm at the first alarm frequency.

[0151] S1053. If the identification result of the operation identification indicates that the probability of a collision occurring within the target time period is greater than or equal to the second threshold, control the loading and unloading equipment to issue an audible alarm at the second alarm frequency and increase the control resistance of the loading and unloading equipment; the second alarm frequency is greater than the first alarm frequency.

[0152] In this embodiment of the application, if there is a work hazard, the target work vehicle is controlled. For example, the collision probability is determined by the prediction result. If the first threshold ≤ the collision probability < the second threshold, an audible alarm is triggered. If the second threshold < the collision probability, the frequency of the audible alarm is increased, and the control resistance of the target work vehicle is increased.

[0153] In one implementation, a mapping relationship can exist between distance and collision probability. For example, a mapping relationship can be established based on the distance between the vehicle and the obstacle, converting distance into collision probability; the closer the distance, the higher the collision probability. By setting different collision probability thresholds, potential collision risks can be stratified for early warning. When the collision probability is within a low threshold range, only an audible alarm is triggered to alert the driver; however, as the collision probability increases further, not only is the frequency of the audible alarm increased, but resistance is also increased on the accelerator and steering wheel of the work vehicle, forcibly reducing the vehicle's speed and steering agility, thereby effectively reducing the possibility of a collision.

[0154] In one implementation, increasing control resistance on the target vehicle can include increasing control resistance on the accelerator and steering wheel. For example, if the driver turns right, the resistance to turning the steering wheel to the right is increased, making steering and forward movement difficult (achieved by increasing the control resistance of the accelerator), thereby reducing the possibility of a collision. This resistance is gradually increased until the target vehicle comes to a stop.

[0155] Based on the same inventive concept, this application also provides a warehouse loading and unloading equipment operation identification device for implementing the above-mentioned warehouse loading and unloading equipment operation identification method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more warehouse loading and unloading equipment operation identification device embodiments provided below can be found in the above-described limitations of the warehouse loading and unloading equipment operation identification method, and will not be repeated here.

[0156] In one exemplary embodiment, Figure 6 This is a schematic diagram of the operation identification device for loading and unloading equipment in a warehouse provided in some embodiments, such as... Figure 6 As shown, the operation identification device 600 for loading and unloading equipment in the warehouse includes:

[0157] The data acquisition module 601 is used to acquire, from the target point cloud dataset collected by loading and unloading equipment in the warehouse, the dataset to be enhanced in the area with collision risk and the dataset to be optimized in the area without collision risk.

[0158] The obstacle detection module 602 is used to perform obstacle detection on the dataset to be enhanced and obtain obstacle detection results;

[0159] The lightweight processing module 603 is used to perform lightweight processing on the dataset to be optimized, so as to obtain the optimized dataset.

[0160] The trajectory prediction module 604 is used to input both the obstacle detection results and the optimized dataset into the collision avoidance model to obtain the predicted trajectory of the loading and unloading equipment during the target time period.

[0161] The identification control module 605 is used to identify the operation of the loading and unloading equipment based on the predicted operating trajectory, and to control the operation of the loading and unloading equipment based on the identification results.

[0162] In some embodiments, the data acquisition module includes a data acquisition unit, a dataset determination unit, and a dataset determination unit. The acquisition unit is used to acquire initial point cloud datasets from multiple target areas collected by the loading and unloading equipment. The multiple target areas are areas within multiple different azimuth angle ranges collected by the loading and unloading equipment, and the union of these different azimuth angle ranges is the omnidirectional angle. The dataset determination unit is used to identify point cloud data in each collision-risk area within the initial point cloud dataset of each target area, and to stitch together the point cloud data in each collision-risk area to obtain the dataset to be enhanced. The dataset determination unit is used to determine the point cloud data outside the point cloud data in each collision-risk area within the initial point cloud dataset of each target area as the dataset to be optimized.

[0163] In some embodiments, the obstacle detection module includes a sampling unit, a model determination unit, and an obstacle detection unit. The sampling unit is used to sample a preset number of point cloud data from the dataset to be reinforced using the farthest point sampling algorithm to obtain sampled point cloud data. The model determination unit is used to determine an updated obstacle detection model based on the sampled point cloud data. The obstacle detection unit is used to perform obstacle detection on the dataset to be reinforced using the updated obstacle detection model to obtain the obstacle detection result.

[0164] In some embodiments, the model determination unit is further configured to determine the ratio of the number of obstacle point cloud data in the sampled point cloud data to the total number of sampled point cloud data as the obstacle proportion of the sampled point cloud data; determine the target weight of each obstacle category under multiple obstacle categories based on the obstacle proportion of the sampled point cloud data; and update the preset obstacle detection model based on the target weight of each obstacle category to obtain the updated obstacle detection model.

[0165] In some embodiments, the model determination unit is further configured to determine a smoothing coefficient for smoothing the labels based on the obstacle proportion of the sampled point cloud data; smooth the real labels of each obstacle category using the smoothing coefficient to obtain the target labels of each obstacle category; perform obstacle detection on the dataset to be enhanced using a preset obstacle detection model to obtain the probability of the dataset to be enhanced in each obstacle category; determine the loss value of each obstacle category based on the target labels of each obstacle category and the probability of each obstacle category; and determine the target weight of each obstacle category based on the loss value of each obstacle category.

[0166] In some embodiments, the lightweight processing module includes a segmentation unit, an identification unit, and a lightweight processing unit. The segmentation unit is used to segment the point cloud data in the dataset to be optimized to obtain multiple sub-data sets in the segmented regions. The identification unit is used to identify at least one sub-data set in a non-edge region from the sub-data sets in the multiple segmented regions. The lightweight processing unit is used to replace the sub-data sets in each non-edge region with the centroid coordinate data of each non-edge region to obtain the optimized dataset.

[0167] In some embodiments, the identification unit is further configured to determine the normal vector between each target point and multiple neighboring points in the subset of each segmented region; determine the average angle of the subset of each segmented region based on the normal vector; and identify the subset of each segmented region corresponding to the average angle that is less than or equal to a preset angle as the subset of each non-edge region.

[0168] In some embodiments, the identification control module includes an identification unit and a control unit. The identification unit is used to identify the loading and unloading equipment based on the predicted operating trajectory and obtain the identification result of the operation. The control unit is used to control the loading and unloading equipment to sound an alarm at a first alarm frequency when the identification result of the operation indicates that the probability of a collision occurring within a target time period is greater than or equal to a first threshold and less than a second threshold. When the identification result of the operation indicates that the probability of a collision occurring within a target time period is greater than or equal to the second threshold, the control unit is used to control the loading and unloading equipment to sound an alarm at a second alarm frequency and increase the control resistance of the loading and unloading equipment. The second alarm frequency is greater than the first alarm frequency.

[0169] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0170] The modules in the operation identification device for loading and unloading equipment in the aforementioned warehouse can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0171] In one exemplary embodiment, Figure 7 This is a schematic diagram of the structure of a computer device provided for some embodiments. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wireless Fidelity (WIFI), mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for identifying the operation of loading and unloading equipment in a warehouse. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0172] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] For example, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method of any of the above embodiments.

[0174] For example, in an exemplary embodiment, when the processor executes a computer program, it performs the following: obtaining a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk from a target point cloud dataset collected from the loading and unloading equipment in the warehouse; performing obstacle detection on the dataset to be enhanced to obtain obstacle detection results; performing lightweight processing on the dataset to be optimized to obtain an optimized dataset; inputting both the obstacle detection results and the optimized dataset into an anti-collision model to obtain a predicted operating trajectory of the loading and unloading equipment during a target time period; performing operation identification on the loading and unloading equipment based on the predicted operating trajectory; and controlling the operation of the loading and unloading equipment based on the operation identification results.

[0175] In one embodiment, a computer-readable storage medium is provided, wherein a computer program, when executed by a processor, implements the steps of the method provided in any of the above embodiments.

[0176] For example, in one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When executed by a processor, the computer program performs the following steps: obtaining a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk from a target point cloud dataset collected by loading and unloading equipment in a warehouse; performing obstacle detection on the dataset to be enhanced to obtain obstacle detection results; performing lightweight processing on the dataset to be optimized to obtain an optimized dataset; inputting both the obstacle detection results and the optimized dataset into an anti-collision model to obtain a predicted operating trajectory of the loading and unloading equipment during a target time period; performing operation identification on the loading and unloading equipment based on the predicted operating trajectory; and controlling the operation of the loading and unloading equipment based on the operation identification results.

[0177] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method provided in any of the above embodiments.

[0178] For example, in one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: obtaining a dataset to be enhanced in areas with collision risk and a dataset to be optimized in areas without collision risk from a target point cloud dataset collected from loading and unloading equipment in a warehouse; performing obstacle detection on the dataset to be enhanced to obtain obstacle detection results; performing lightweight processing on the dataset to be optimized to obtain an optimized dataset; inputting both the obstacle detection results and the optimized dataset into an anti-collision model to obtain a predicted operating trajectory of the loading and unloading equipment during a target time period; performing operation identification on the loading and unloading equipment based on the predicted operating trajectory, and controlling the operation of the loading and unloading equipment based on the operation identification results.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the methods described above.

[0180] The processor, functional modules, or functional units in any embodiment of this application may include an integration of one or more of the following: a general-purpose processor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU), a controller, a microcontroller, a microprocessor, a programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, a quantum computing-based data processing logic unit, an artificial intelligence (AI) processor, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0181] The memory or computer-readable storage medium in any embodiment of this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory includes integration of one or more of the following: Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory, Optical Disc, Compact Disc Read-Only Memory (CD-ROM), Magnetic Tape, Floppy Disk, Flash Memory, Optical Memory, High-Density Embedded Non-Volatile Memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), Graphene Memory, Volatile Memory, etc. Volatile memory includes one or more of the following: Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0182] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0183] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for identifying the operation of loading and unloading equipment in a warehouse, characterized in that, The method includes: From the target point cloud dataset collected by the loading and unloading equipment in the warehouse, obtain the dataset to be strengthened in the area with collision risk and the dataset to be optimized in the area without collision risk; Obstacle detection is performed on the dataset to be enhanced to obtain obstacle detection results; The dataset to be optimized is subjected to lightweight processing to obtain the optimized dataset; The obstacle detection results and the optimized dataset are both input into the collision avoidance model to obtain the predicted operating trajectory of the loading and unloading equipment during the target time period; Based on the predicted operating trajectory, the distance between the loading and unloading equipment and the nearest obstacle within the target time period is determined; Based on the distance between the loading and unloading equipment and the nearest obstacle during the target time period, the identification result of the operation identification of the loading and unloading equipment is determined, and the identification result of the operation identification represents the probability of a collision occurring during the target time period; The operation of the loading and unloading equipment is controlled based on the identification results of the operation identification.

2. The method according to claim 1, characterized in that, The process of obtaining the target point cloud dataset collected from the loading and unloading equipment in the warehouse, including the dataset to be enhanced in areas with collision risk and the dataset to be optimized in areas without collision risk, includes: Obtain an initial point cloud dataset from multiple target areas collected by the loading and unloading equipment; wherein, the multiple target areas are areas within multiple different azimuth angle ranges collected by the loading and unloading equipment respectively, and the union of the multiple different azimuth angle ranges is the omnidirectional angle; Identify the point cloud data in each region with collision risk in the initial point cloud dataset of each target region, and stitch together the point cloud data in each region with collision risk to obtain the dataset to be enhanced. The point cloud data outside the point cloud data in the regions with collision risks in the initial point cloud dataset of each target region is determined as the dataset to be optimized.

3. The method according to claim 1 or 2, characterized in that, The obstacle detection process on the dataset to be enhanced, to obtain obstacle detection results, includes: The farthest point sampling algorithm is used to sample a preset number of point cloud data from the dataset to be enhanced, resulting in sampled point cloud data. Based on the sampled point cloud data, determine the updated obstacle detection model; The updated obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the detection results of the obstacles are obtained.

4. The method according to claim 3, characterized in that, The step of determining the updated obstacle detection model based on the sampled point cloud data includes: The ratio of the number of obstacle point cloud data to the total number of sampling point cloud data is determined as the obstacle proportion of the sampling point cloud data. Based on the obstacle proportion of the sampled point cloud data, determine the target weight of each obstacle category under multiple obstacle categories; The preset obstacle detection model is updated according to the target weight of each obstacle category to obtain the updated obstacle detection model.

5. The method according to claim 4, characterized in that, The step of determining the target weight of each obstacle category under multiple obstacle categories based on the obstacle proportion of the sampled point cloud data includes: Based on the obstacle ratio of the sampled point cloud data, a smoothing coefficient is determined for smoothing the labels; The smoothing coefficient is used to smooth the true labels of each obstacle category to obtain the target labels of each obstacle category; The preset obstacle detection model is used to detect obstacles in the dataset to be enhanced, and the probability of the dataset to be enhanced in each obstacle category is obtained. Based on the target label of each obstacle category and the probability of each obstacle category, the loss value of each obstacle category is determined, and based on the loss value of each obstacle category, the target weight of each obstacle category is determined.

6. The method according to claim 1 or 2, characterized in that, The step of performing lightweight processing on the dataset to be optimized to obtain the optimized dataset includes: The point cloud data in the dataset to be optimized is segmented to obtain multiple sub-datasets in the segmented regions; From the subsets of data in the plurality of segmented regions, identify at least one subset of data in a non-edge region; The optimized dataset is obtained by replacing the subsets in each of the non-edge regions with the centroid coordinates of each of the non-edge regions.

7. The method according to claim 6, characterized in that, The step of identifying at least one subset of data from a non-edge region among the subsets of data from the plurality of segmented regions includes: Determine the normal vector between each target point and multiple neighboring points in the subset of data in each segmented region; Based on the normal vector, determine the average included angle of the subsets in each segmented region; The subsets of data in each segmented region corresponding to the average angle that is less than or equal to the preset angle are determined as subsets of data in each non-edge region.

8. The method according to claim 1 or 2, characterized in that, The control of the operation of the loading and unloading equipment based on the identification results of the operation identification includes: If the identification result of the operation identification indicates that the probability of a collision occurring within the target time period is greater than or equal to a first threshold and less than a second threshold, the loading and unloading equipment is controlled to issue an audible alarm at a first alarm frequency. If the identification result of the operation identification indicates that the probability of a collision occurring within the target time period is greater than or equal to a second threshold, the loading and unloading equipment is controlled to issue an audible alarm at a second alarm frequency, and the control resistance of the loading and unloading equipment is increased; the second alarm frequency is greater than the first alarm frequency.

9. A work identification device for loading and unloading equipment in a warehouse, characterized in that, The device includes: The data acquisition module is used to acquire, from the target point cloud dataset collected by the loading and unloading equipment in the warehouse, the dataset to be enhanced in the area with collision risk and the dataset to be optimized in the area without collision risk. An obstacle detection module is used to perform obstacle detection on the dataset to be enhanced and obtain obstacle detection results; The lightweight processing module is used to perform lightweight processing on the dataset to be optimized to obtain the optimized dataset. The trajectory prediction module is used to input the obstacle detection results and the optimized dataset into the collision avoidance model to obtain the predicted trajectory of the loading and unloading equipment during the target time period. The identification and control module is used to determine the distance between the loading and unloading equipment and the nearest obstacle in the target time period based on the predicted operating trajectory; determine the identification result of the operation identification of the loading and unloading equipment based on the distance between the loading and unloading equipment and the nearest obstacle in the target time period, wherein the identification result of the operation identification represents the probability of a collision occurring in the target time period; and control the operation of the loading and unloading equipment based on the identification result of the operation identification.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images