Cargo placement control method and device, computer device and storage medium

Abstract

The application relates to a cargo placement control method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a warehouse location map by acquiring image collection of a warehouse location; the warehouse location objects in the warehouse location map are at least one; the warehouse location represented by the warehouse location objects is used for storing cargos; the warehouse location objects in the warehouse location map are positioned and detected; the cargo placement state of the warehouse location objects is recognized, and the cargo placement state of the warehouse location represented by the warehouse location objects is obtained; the cargo placement state is used for indicating whether there is a cargo on the warehouse location represented by the warehouse location objects; and the cargo placement control processing of the carrying equipment is performed according to the cargo placement state. The cargo placement state of the warehouse location objects in the warehouse location map can be recognized, and whether there is a cargo on the warehouse location represented by the warehouse location objects can be automatically judged through programming, so that the carrying equipment can timely place the cargos, and the efficiency of the cargo placement is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of logistics application, in particular to a cargo placement control method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of the logistics industry, the number of goods is increasing, and the demand for cargo placement is also increasing. At present, the goods on the storage location on the loading platform need to be placed by the carrying device, and then the goods on the storage location are manually carried to the carriage. After all the goods on the storage location are taken away by the manual operation, the carrying device needs to place the next goods on the storage location.

[0003] However, the carrying device itself cannot determine whether the goods on the storage location have been completely taken away, so that the carrying device cannot place the goods in time, thereby affecting the efficiency of cargo placement. SUMMARY

[0004] Therefore, it is necessary to provide a cargo placement control method and device, computer equipment, computer readable storage medium and computer program product capable of improving the efficiency of cargo placement.

[0005] In a first aspect, the present application provides a cargo placement control method. The method comprises:

[0006] obtaining a storage location image obtained by image acquisition on the storage location; the storage location object in the storage location image is at least one; the storage location represented by the storage location object is used for storing goods;

[0007] locating and detecting the storage location object in the storage location image;

[0008] identifying the cargo placement state of the storage location object, obtaining the cargo placement state of the storage location represented by the storage location object; the cargo placement state is used to indicate whether there is cargo on the storage location represented by the storage location object;

[0009] controlling the cargo placement of the carrying device according to the cargo placement state.

[0010] In a second aspect, the present application further provides a cargo placement control device. The device comprises:

[0011] an image acquisition module for obtaining a storage location image obtained by image acquisition; the storage location object in the storage location image is at least one; the storage location represented by the storage location object is used for storing goods;

[0012] a positioning and detection module for locating and detecting the storage location object in the storage location image;

[0013] The state recognition module is configured to recognize the cargo placement state of the bin object, and obtain a cargo placement state of a bin represented by the bin object.

[0014] The placement control module is configured to control the cargo placement of the handling device according to the cargo placement state.

[0015] In some embodiments, the bin map is acquired by an image acquisition device, and the cargo placement control device further comprises a device installation module configured to calculate a field of view range of the image acquisition device according to the highest cargo height, device parameters of the image acquisition device, and a preset installation position; if the cargo on a target bin can be completely displayed in the field of view range, the image acquisition device is installed according to the preset installation position; wherein the target bin is a bin represented by each bin object in the bin map.

[0016] In some embodiments, the positioning detection module is further configured to obtain a pre-configuration file, and obtain bin information of the bin object from the pre-configuration file; the bin information is obtained by calibrating a sample bin map; the sample bin map is obtained by pre-acquiring images of the bin represented by the bin object by the image acquisition device; and the bin object in the bin map is obtained by performing positioning detection on the bin map according to the bin information.

[0017] In some embodiments, the cargo placement state is obtained by recognizing the cargo placement state of the cargo object by a cargo state recognition engine; the cargo state recognition engine is converted from a cargo state recognition model trained by deep learning; and the detection inference speed of the cargo state recognition engine is superior to that of the cargo state recognition model.

[0018] In some embodiments, the placement control module is further configured to, if the cargo placement state is a no-cargo state, generate a cargo operation signal to make the handling device place cargo in the bin represented by the bin object according to the cargo operation signal.

[0019] In some embodiments, the placement control module is further configured to, if the cargo placement state is a cargo-present state, generate a notification prompting that the bin represented by the bin object has cargo.

[0020] In some embodiments, the placement control module is further configured to, if the storage location represented by the plurality of storage location objects is in a no- goods state, generate a next goods operation signal after the handling device places goods in a last storage location in the no-goods state, and control the handling device to place goods in a next storage location corresponding to the no-goods state based on the next goods operation signal.

[0021] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the above-mentioned goods placement control method when executing the computer program.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the above-mentioned goods placement control method.

[0023] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the steps of the above-mentioned goods placement control method.

[0024] The above-mentioned goods placement control method, device, computer device, storage medium, and computer program product can obtain a storage location image collected by image collection equipment for a storage location. The storage location objects in the storage location image are at least one. The storage location represented by the storage location objects is used for storing goods. The storage location objects in the storage location image are detected. The storage location objects are identified to obtain a goods placement state of the storage location represented by the storage location objects. The goods placement state is used to indicate whether the storage location represented by the storage location objects has goods. The handling device is controlled based on the goods placement state. The storage location objects in the storage location image are identified to automatically determine whether the storage location represented by the storage location objects has goods, so that the handling device can timely place goods, and the efficiency of goods placement is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 An application environment diagram of the goods placement control method in some embodiments;

[0026] Figure 2 A flowchart of the goods placement control method in some embodiments;

[0027] Figure 3 An installation position diagram of the image collection equipment in some embodiments;

[0028] Figure 4 A flowchart of the goods placement control method in some other embodiments;

[0029] Figure 5 Structure block diagram of the goods placement control device in some embodiments;

[0030] Figure 6 Internal structure diagram of the computer device in some embodiments. DETAILED DESCRIPTION

[0031] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0032] The goods placement control method provided by the embodiments of the present application can be applied to Figure 1 the application environment as shown. Among them, the image acquisition device 102 used for image acquisition and the server 104 need to be connected to the same network to ensure that the server 104 can access all image acquisition devices 102, and the server 104 communicates with the image acquisition device 102 through the network. The image acquisition device 102 acquires images for the storage location of the goods, obtains the storage location map, and the image acquisition device 102 can also send the acquired storage location map to the server 104 for goods placement control. Specifically, after obtaining the storage location map, the server 104 locates the storage location object in the storage location map, and identifies the goods placement state of the storage location object, to obtain the goods placement state of the storage location represented by the storage location object, wherein the goods placement state is used to indicate whether there is goods on the storage location represented by the storage location object. In addition, the server 104 also performs goods placement control processing on the handling equipment according to the goods placement state. The number of image acquisition devices 102 can be one or more, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0033] In some embodiments, as Figure 2 shown, a goods placement control method is provided, which can be applied to the server, or realized through the interaction between the server and the image acquisition device, and the embodiments of the present application do not make specific limitations. Taking the case that the method is applied to the server in Figure 1 , the method comprises the following steps:

[0034] Step 202, obtaining a storage location map obtained by image acquisition for a storage location.

[0035] Among them, the storage location refers to the specific storage location of the goods in the warehouse. It can be understood that the shape of the storage location can be rectangular or other shapes, and the present application does not limit the specific shape of the storage location.

[0036] The warehouse position map is obtained by image acquisition of one or more warehouse positions by an image acquisition device. The warehouse position object in the warehouse position map is at least one, and the warehouse position object is used to represent the warehouse position in the warehouse position map. It can be understood that the warehouse position represented by the warehouse position object is used to store goods. It can be understood that the warehouse position map does not only include the image content of the warehouse position, but also can include the image content of other position areas, as long as the image content of the warehouse position in the warehouse position map is ensured.

[0037] The image acquisition device refers to a device with a photographing function, which can be at least one of various cameras and mobile devices.

[0038] The other position area can include a safety area, which refers to an area located around the warehouse position for alarm prompt. For example, it can be identified whether a person or a handling device is included in the safety area, and the number of persons or the number of handling devices in the safety area can also be identified. If a person or a handling device is identified to enter the set safety area range, an alarm prompt can also be generated. It should be noted that the shape of the safety area can be polygonal, and the specific number of vertices, area and position of the safety area can be set according to actual needs.

[0039] Specifically, the image acquisition device performs image acquisition on the warehouse position for storing goods to obtain the warehouse position map. The server obtains the warehouse position map obtained by the image acquisition device for image acquisition, and is used for subsequent positioning detection of the warehouse position object in the warehouse position map.

[0040] Step 204, positioning detection of the warehouse position object in the warehouse position map.

[0041] Specifically, the server can identify the warehouse position object in the warehouse position map which needs to be identified for the goods placement state. It can be understood that the warehouse position object which needs to be identified for the goods placement state can be all or part of the warehouse position objects in the warehouse position map.

[0042] In some embodiments, the server can locate the warehouse position object from the warehouse position map by using the warehouse position information pre-configured for calibrating the warehouse position.

[0043] In other embodiments, the server can automatically perform image segmentation processing on the warehouse position map to locate and detect the warehouse position object. It can be understood that the specific implementation of how to locate the warehouse position object from the warehouse position map is not limited in the present application.

[0044] Step 206, identifying the goods placement state of the warehouse position object to obtain the goods placement state of the warehouse position represented by the warehouse position object.

[0045] The goods placement state identification refers to identifying whether there is goods on the warehouse position represented by the warehouse position object, and the goods placement state is used to indicate whether there is goods or no goods on the warehouse position represented by the warehouse position object.

[0046] In some embodiments, the server can directly perform the cargo placement state recognition on the storage location object through the pre-trained cargo state recognition model to determine whether there is cargo on the storage location represented by the storage location object.

[0047] In other embodiments, the server can also perform the cargo placement state recognition on the storage location object after optimizing the inference of the cargo state recognition model to determine whether there is cargo on the storage location represented by the storage location object.

[0048] The cargo state recognition model refers to a neural network model trained using a deep learning algorithm to recognize the placement state of the cargo. It can be understood that the cargo state recognition model can be a multi-classification model, i.e., the cargo state recognition model is not limited to detecting and recognizing the placement state of the cargo, but can also recognize objects such as people and forklifts.

[0049] Deep learning refers to learning the internal rules and representation levels of sample data, and these learning processes can recognize data such as text, images, and sound.

[0050] Step 208, performing cargo placement control processing on the handling equipment according to the cargo placement state.

[0051] The handling equipment is a device for transporting and placing cargo, and the handling equipment can include at least one of an automated guided vehicle (AGV) and a forklift.

[0052] Specifically, the server determines whether the storage location represented by the storage location object has cargo according to the cargo placement state, and performs cargo placement control processing on the handling equipment in the case of having or not having cargo in the corresponding storage location.

[0053] In some embodiments, if the server determines that the cargo placement state of the storage location represented by the storage location object is a no-cargo state, the handling equipment is controlled to place the cargo on the storage location.

[0054] In other embodiments, if the server determines that the cargo placement state of the storage location represented by the storage location object is a cargo state, the server reports that there is cargo on the corresponding storage location.

[0055] In the above cargo placement control method, a warehouse location map is obtained by image acquisition of the warehouse locations; the warehouse location objects in the warehouse location map are at least one; the warehouse locations represented by the warehouse location objects are used for storing cargos; the warehouse location objects in the warehouse location map are detected; the cargo placement state of the warehouse location objects is identified to obtain the cargo placement state of the warehouse locations represented by the warehouse location objects; the cargo placement state is used to indicate whether there is a cargo on the warehouse locations represented by the warehouse location objects; and the cargo placement control of the handling equipment is performed according to the cargo placement state. By identifying the cargo placement state of the warehouse location objects in the warehouse location map, it can be determined by a program whether there is a cargo on the warehouse locations represented by the warehouse location objects, so as to control the handling equipment to place the cargos in time, thereby improving the efficiency of cargo placement.

[0056] In some embodiments, the warehouse location map is acquired by an image acquisition device, and before step 202, the cargo placement control method further includes a step of installing the image acquisition device. The step of installing the image acquisition device includes: calculating the field of view range of the image acquisition device according to the maximum cargo height, the device parameters of the image acquisition device, and a preset installation position; and if the cargo on the target warehouse location can be completely displayed in the field of view range, installing the image acquisition device according to the preset installation position.

[0057] The target warehouse location is the warehouse location corresponding to the warehouse location represented by each warehouse location object in the warehouse location map in the real world.

[0058] The maximum cargo height refers to the maximum height at which the cargos on the warehouse locations can be stacked.

[0059] The device parameters of the image acquisition device include, but are not limited to, the focal length of the image acquisition device, such as 2.8 mm, 4 mm, and 6 mm, etc.

[0060] The preset installation position includes, but is not limited to, at least one of the preset installation height and the preset installation angle of the image acquisition device.

[0061] Specifically, according to the focal length of the image acquisition device, the horizontal and vertical angles corresponding to the focal length can be directly determined, for example, the horizontal angle corresponding to the focal length of 4 mm is 70 degrees, and the vertical angle is 40 degrees. According to the maximum cargo height, the horizontal and vertical angles, and the preset installation height, the field length and the field width of the image acquisition device at the preset installation height can be calculated, and the field of view range of the image acquisition device at the preset installation height can be calculated according to the field length and the field width. The specific process of the field length and the field width is shown in formula (1) and formula (2):

[0062] L1 = 2 * (H - h) * tan (A / 2) (1)

[0063] L2 = 2 * (H - h) * tan (B / 2) (2)

[0064] Where L1 refers to the field of view length, L2 refers to the field of view width, H refers to the preset installation height, h refers to the maximum cargo height, A refers to the horizontal viewing angle, and B refers to the vertical viewing angle.

[0065] After calculating the field of view length and field of view width using formulas (1) and (2) respectively, multiplying the field of view length and field of view width will give the field of view range that the image acquisition device can see at the preset installation height.

[0066] In some embodiments, after calculating the field of view, if the goods in the target storage location can be fully displayed within the field of view, then the image acquisition device is installed according to a preset installation position. Specifically, the field of view can be compared with a preset field of view; if the field of view is greater than or equal to the preset field of view, it is determined that the goods in the target storage location can be fully displayed within the field of view. The preset field of view can be determined based on the spatial size of the target storage location, specifically based on the storage area of ​​the target storage location.

[0067] In practical applications, such as Figure 3 As shown, if the target storage location is a single location and can only hold two goods, the visible area (i.e., the field of view) of the image acquisition device at the preset installation height is calculated based on the maximum goods height h, the preset installation height H, and the horizontal and vertical viewing angles of the image acquisition device. If the preset field of view is set to the storage area of ​​one location corresponding to the target storage location, and the calculated field of view is equal to the preset field of view, then the image acquisition device can completely see all the goods in the target storage location within its field of view. Figure 3 The presence of two items indicates that the preset installation height is reasonable, and the image acquisition device can be installed directly according to the preset installation height H.

[0068] In some embodiments, if the goods in the target storage location cannot be fully displayed within the field of view, the preset installation position is adjusted until the adjusted preset installation position enables the goods in the target storage location to be fully displayed within the field of view; the image acquisition device is then installed according to the adjusted preset installation position.

[0069] Specifically, if the goods in the target storage location cannot be fully displayed within the field of view, it indicates that the field of view of the image acquisition device is smaller than the preset field of view, and the preset installation position is not set appropriately. In this case, the preset installation position needs to be adjusted. Specifically, the preset installation height and preset installation angle of the image acquisition device can be adjusted. Install the image acquisition device according to the adjusted preset installation height and preset installation angle so that the image acquisition device can fully see all the goods in the target storage location within its field of view.

[0070] It can be understood that, affected by lens manufacturing precision, the warehouse location image captured by the image acquisition device can have different degrees of distortion. In order to improve the recognition effect of image shooting, it can be considered to calibrate the image acquisition device internal parameter before installing the image acquisition device, so as to remove the distortion of the image acquisition device.

[0071] Specifically, the image acquisition device captures the calibration board to obtain a calibration board image. The calibration board image is imported into the calibration software, and the calibration software calibrates the calibration board image to obtain the image acquisition device internal parameter and the distortion parameter. The image acquisition device internal parameter and the distortion parameter are updated to the configuration file of the image acquisition device to complete the process of removing the distortion of the image acquisition device.

[0072] In actual application, it can be considered to install the image acquisition device using a fixing device vertically or obliquely above the loading platform. The installation position of the image acquisition device needs to ensure that it covers all the warehouse locations and can see all the goods. After all the image acquisition devices are installed, the number of installed image acquisition devices can be calculated. Among them, it is necessary to ensure that each image acquisition device captures as many warehouse locations as possible to reduce the number of installed image acquisition devices. In this way, not only can the goods placement state in multiple warehouse locations be automatically recognized through the warehouse location image captured by a single image acquisition device, but also the installation cost of the image acquisition device can be reduced.

[0073] In some embodiments, step 204 specifically includes but is not limited to: obtaining a pre-configuration file, obtaining warehouse location information of a warehouse location object from the pre-configuration file; and performing positioning detection on the warehouse location image according to the warehouse location information to obtain the warehouse location object in the warehouse location image.

[0074] The warehouse location information is obtained by calibrating a sample warehouse location image, and the sample warehouse location image is obtained by pre-image acquisition of the warehouse location object by the image acquisition device.

[0075] The pre-configuration file is used to set the warehouse location information of the warehouse location object, and draw the warehouse location object in the warehouse location image captured by the image acquisition device to represent the specific position of the warehouse location object in the warehouse location image.

[0076] It can be understood that the warehouse location information includes but is not limited to the spatial size of the warehouse location (such as the length and width of the warehouse location), the warehouse location number, and the specific position of the warehouse location object in the warehouse location image. Among them, the specific position of the warehouse location object in the warehouse location image refers to the pixel coordinates of each vertex of the warehouse location object in the pixel coordinate system, etc.

[0077] Specifically, the server obtains the pre-configuration file, obtains the specific position of the bin object in the bin map from the pre-configuration file, that is, the pixel coordinates of each vertex of the bin represented by the bin object recorded in the pre-configuration file in the pixel coordinate system. The server can locate the bin object in the bin map by positioning detection of the bin map according to the pixel coordinates of each vertex.

[0078] In actual application, the configuration process of the pre-configuration file can be: collecting multiple bin maps of multiple bins by an image acquisition device, and ensuring that each bin map is not repeated. The bin area is set in the configuration file of the cargo state recognition program, specifically, the length, width, bin number (which is one-to-one corresponding to the actual bin number) of each bin in the bin area, and the coordinate point of each bin in the pixel coordinate system are input in the configuration file to generate the pre-configuration file. It should be noted that the bin area includes all bins, and the cargo state recognition program is an executable program integrated with a tensorrt environment (i.e., a deployed cargo state recognition engine), which can detect and recognize the target. For example, when there is cargo in a bin, the cargo state recognition program will highlight the bin with cargo through the detection box in the recognition interface.

[0079] In some embodiments, the cargo placement state is obtained by cargo placement state recognition of the cargo object by the cargo state recognition engine, which can be integrated and deployed on the server. The cargo state recognition engine is converted from a cargo state recognition model trained by deep learning, and the detection inference speed of the cargo state recognition engine is better than that of the cargo state recognition model. The cargo state recognition model established by the deep learning algorithm can automatically find accurate features, thereby improving the accuracy of cargo state recognition. Converting the cargo state recognition model trained by deep learning into a cargo state recognition engine can further optimize inference and further improve the accuracy of cargo state recognition.

[0080] The cargo state recognition engine refers to an inference optimizer that optimizes the trained cargo state recognition model. The cargo state recognition engine can optimize inference and more accurately recognize various cargos, and can also be deployed for acceleration.

[0081] In some embodiments, the training process of the cargo state recognition model is as follows: the installed image acquisition device and the corresponding server are connected to the same network to ensure that the server can access all image acquisition devices. The image acquisition device is opened by the server to acquire images, and the image acquisition device acquires images of different goods in different storage locations, and selects non-repeated images as cargo sample images, thereby ensuring the balance of the training samples, wherein the cargo sample images include but are not limited to images of a single cargo, images of multiple cargos stacked at different intervals, angles and positions, and images of different types of cargos. Then, each cargo sample image is labeled to obtain a labeled image set, wherein the labeled categories include but are not limited to at least one of a person, a forklift and a cargo. For example, if the image content of a cargo sample image includes a person, a forklift and a cargo, the object boxes corresponding to the person, the forklift and the cargo are selected out respectively by using a labeling tool, and corresponding labels are marked, which are used to tell the model what is a person, what is a forklift and what is a cargo during model training. Finally, the labeled image set is placed in the original neural network model for training to obtain optimal model parameters, and the neural network model is updated according to the optimal model parameters to obtain the trained cargo state recognition model.

[0082] In some embodiments, the original neural network model can select a yolov5 model, wherein the yolov5 model is a neural network model trained by using a single-stage target detection algorithm. The cargo state recognition model trained according to the yolov5 model has fast inference speed, small space occupation and higher accuracy.

[0083] In some embodiments, the process of converting the cargo state recognition model into a cargo state recognition engine is as follows: the format of the model file corresponding to the trained cargo state recognition model is converted into the file format corresponding to the cargo state recognition engine to generate an engine file. Specifically, the cargo state recognition model is imported into the cargo state recognition engine to generate an engine file, and the engine file is serialized and saved, and then it can be conveniently and quickly called to execute the accelerated inference of the cargo state recognition model.

[0084] In actual application, tensorrt can be selected as the cargo state recognition engine of the present application, wherein tensorrt is a high-performance deep learning inference optimizer and engine.

[0085] It can be understood that in some embodiments, the process of converting the cargo state recognition model into the cargo state recognition engine (i.e., tensorrt) is as follows: obtaining the model file in.pt format corresponding to the cargo state recognition model, converting the model file in.pt format into a model file in.wts format. Put the model file in.wts format into the folder corresponding to tensorrt, double-click to run the engine generation program in the folder to generate a new engine file. Copy the new engine file and replace the original engine file to generate the cargo state recognition engine.

[0086] In some embodiments, step 208 specifically includes but is not limited to: if the cargo placement state is no cargo state, generating a cargo operation signal to make the handling equipment place the cargo in the storage location represented by the storage location object according to the cargo operation signal. The cargo operation signal is automatically generated by programming, thereby controlling the handling equipment to place the cargo in time, thereby improving the efficiency of cargo placement.

[0087] In some embodiments, the cargo placement control method specifically further includes but is not limited to: if the cargo placement state is the cargo state, generating a notification for prompting that the storage location represented by the storage location object has cargo, indicating that the handling equipment does not need to place cargo in the storage location at this time.

[0088] In some embodiments, the cargo placement control method specifically further includes but is not limited to: if the cargo placement state of the storage locations represented by the plurality of storage location objects is no cargo state, after the handling equipment places the cargo in the last storage location in the no cargo state, generating the next cargo operation signal to control the handling equipment to place the cargo in the next storage location corresponding to the no cargo state based on the next cargo operation signal, so that the handling equipment can perform continuous loading work.

[0089] In actual application, the cargo state recognition program is run on the server to monitor all storage locations in the storage location area. In the program interface of the cargo state recognition program, when the cargo enters the detection area (i.e., the storage location area), the cargo state recognition program will automatically label the detection frame on the cargo object and display the corresponding cargo label. When the cargo is recognized to be stored in the storage location, the cargo state recognition program will report that there is cargo in the corresponding storage location. When no cargo is recognized in the storage location, data in a specific format (e.g., json format) is generated, and the obtained data is transmitted to the system program through a data interface (e.g., JSON-RPC interface). The system program generates a cargo operation signal according to the received data and transmits the cargo operation signal to the handling equipment. The handling equipment will place the cargo in the corresponding storage location after receiving the cargo operation signal.

[0090] It should be noted that the data form in json format can be in the form of key-value pair, for example, setting goods = 1, no goods = 0, and the specific location corresponding to the location number, etc. After the system program receives the data in the form of key-value pair, a goods operation signal is generated, which tells the handling equipment that there is no goods in a certain location, so that the handling equipment goes to the corresponding location to put goods. In addition, after the handling equipment completes a location to put goods, the system program will send the next goods operation signal to the handling equipment, so that the handling equipment goes to the next location to put goods, until all the goods in the location are placed.

[0091] In some embodiments, as shown in Figure 4 The goods placement control method of the present application specifically further includes but is not limited to the following steps:

[0092] Step 402, determine the device parameters and installation information of the image acquisition device.

[0093] Step 404, distortion removal operation is performed on the image acquisition device.

[0094] Step 406, the image acquisition device is fixedly installed above the loading platform according to the installation information.

[0095] Step 408, the location map of the location is acquired by the image acquisition device.

[0096] Step 410, calibrate the location according to the location map, and write the configuration file of the goods state recognition program.

[0097] Step 412, run the goods state recognition program to acquire the sample location map.

[0098] Step 414, annotate the sample location map.

[0099] Step 416, model training and conversion are performed through the annotated sample location map to obtain a goods state recognition model.

[0100] Step 418, generate a goods state recognition engine according to the goods state recognition model.

[0101] Step 420, replace the engine of the source file according to the goods state recognition engine.

[0102] Step 422, run the goods state recognition program again to monitor all locations.

[0103] Step 424, identify whether there is goods in the location through the goods state recognition program, if there is goods, execute step 426, if there is no goods, execute step 428.

[0104] Step 426, the goods state recognition program reports that the location has goods.

[0105] Step 428, the goods state recognition program transmits a put-away signal to the handling equipment.

[0106] Specifically, the device parameters of the image acquisition device are determined, and the installation information of the image acquisition device is determined, wherein the device parameters of the image acquisition device include a focal length, and the installation information of the image acquisition device includes an installation height and an installation angle of the image acquisition device. After the image acquisition device is de-distorted, the image acquisition device is fixedly installed above a loading platform, wherein the loading platform refers to an area for putting goods, and the loading platform is provided with a plurality of pallets (for example, 2). After the image acquisition device is installed, the image acquisition device is used to acquire a bin map for one or more bins, and bin calibration is performed according to the bin map. After calibration, the calibrated bin map is written into a configuration file of the goods state recognition program. After the configuration is completed, the goods state recognition program is run to acquire a sample bin map, and the sample bin map is labeled. The trained model file is converted to obtain a goods state recognition model. The goods state recognition model is converted to generate a goods state recognition engine, and the goods state recognition engine is used to replace an engine of a source file. The goods state recognition program is run again to monitor all bins, and whether there is goods on the bin is recognized by the goods state recognition program. If there is goods, it is reported that the bin has goods. If there is no goods, a put-away signal is transmitted to the handling equipment, so that the handling equipment performs a put-away operation on the bin.

[0107] It should be understood that, although each step in the flowchart involved in each of the above-described embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above-described embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0108] Based on the same inventive concept, the embodiments of the present application also provide a goods placement control device for implementing the above-mentioned goods placement control method. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more goods placement control device embodiments provided below can refer to the limitations of the goods placement control method described above, and will not be described here.

[0109] In some embodiments, as Figure 5As shown, a cargo placement control device is provided, comprising: an image acquisition module 502, a positioning detection module 504, a state recognition module 506, and a placement control module 508, wherein:

[0110] The image acquisition module 502 is configured to acquire a warehouse location map obtained by image collection; the warehouse location objects in the warehouse location map are at least one; and the warehouse locations represented by the warehouse location objects are used for storing cargos.

[0111] The positioning detection module 504 is configured to positionally detect the warehouse location objects in the warehouse location map.

[0112] The state recognition module 506 is configured to recognize the cargo placement state of the warehouse location objects, so as to obtain the cargo placement state of the warehouse locations represented by the warehouse location objects; and the cargo placement state is used to indicate whether there is a cargo on the warehouse locations represented by the warehouse location objects.

[0113] The placement control module 508 is configured to perform cargo placement control processing on the handling equipment according to the cargo placement state.

[0114] According to the above cargo placement control device, the warehouse location map obtained by image collection of the warehouse locations is acquired; the warehouse location objects in the warehouse location map are at least one; the warehouse locations represented by the warehouse location objects are used for storing cargos; the warehouse location objects in the warehouse location map are positionally detected; the cargo placement state of the warehouse location objects is recognized, so as to obtain the cargo placement state of the warehouse locations represented by the warehouse location objects; the cargo placement state is used to indicate whether there is a cargo on the warehouse locations represented by the warehouse location objects; and the cargo placement control processing is performed on the handling equipment according to the cargo placement state. The cargo placement state of the warehouse location objects in the warehouse location map is recognized, so as to automatically determine whether there is a cargo on the warehouse locations represented by the warehouse location objects through programming, thereby controlling the handling equipment to timely place the cargos, and further improving the efficiency of cargo placement.

[0115] In some embodiments, the warehouse location map is collected by an image collection device, and the cargo placement control device further comprises a device installation module, which is configured to calculate the field of view range of the image collection device according to the highest cargo height, the device parameters of the image collection device, and a preset installation position; if the cargos on the target warehouse locations can be completely displayed in the field of view range, the image collection device is installed according to the preset installation position; wherein the target warehouse locations are the warehouse locations represented by the warehouse location objects in the warehouse location map, which correspond to the warehouse locations used for storing cargos in the real world.

[0116] In some embodiments, the positioning detection module 504 is further configured to acquire a pre-configuration file, and obtain the warehouse location information of the warehouse location objects from the pre-configuration file; the warehouse location information is obtained by calibrating a sample warehouse location map; the sample warehouse location map is obtained by pre-image collection of the warehouse locations represented by the warehouse location objects by the image collection device; and the warehouse location objects in the warehouse location map are obtained by positioning detection of the warehouse location map according to the warehouse location information.

[0117] In some embodiments, the goods placement state is obtained by performing goods placement state recognition on the goods object by a goods state recognition engine; the goods state recognition engine is converted from a goods state recognition model trained by deep learning; and the detection inference speed of the goods state recognition engine is superior to that of the goods state recognition model.

[0118] In some embodiments, the placement control module 508 is further configured to, if the goods placement state is the no-goods state, generate a goods operation signal to cause the carrying device to place the goods in the storage location represented by the storage location object according to the goods operation signal.

[0119] In some embodiments, the placement control module 508 is further configured to, if the goods placement state is the goods-present state, generate a notification for prompting that the storage location represented by the storage location object has goods.

[0120] In some embodiments, the placement control module 508 is further configured to, if the goods placement state of the storage locations represented by the plurality of storage location objects is the no-goods state, after the carrying device places the goods in the last storage location in the no-goods state, generate a next goods operation signal to control the carrying device to place the goods in the next storage location corresponding to the no-goods state based on the next goods operation signal.

[0121] The division of each module in the above goods placement control apparatus is only for illustration, and in other embodiments, the goods placement control apparatus can be divided into different modules as needed to complete all or part of the functions of the above goods placement control apparatus.

[0122] Each module in the above goods placement control apparatus can be implemented in whole or in part by software, hardware, and a combination thereof. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0123] In one embodiment, a computer device is provided, which can be a server in Figure 1 , and its internal structure diagram can be as shown in Figure 6As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store image data and storage location information data. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a cargo placement control method.

[0124] Those skilled in the art can understand that, Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or less components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0125] In some embodiments, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0126] In some embodiments, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0127] In some embodiments, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0129] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0130] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A cargo placement control method characterized by, The method comprises: acquiring a warehouse position map obtained by image acquisition on a warehouse position; the warehouse position object in the warehouse position map is at least one; the warehouse position represented by the warehouse position object is used for storing goods; the warehouse position map is acquired by an image acquisition device; the horizontal angle of view, the vertical angle of view of the preset installation height of the image acquisition device, the calculation result of the preset installation height and the highest goods height is the field of view length and the field of view width, the calculation result of the field of view length and the field of view width is the field of view range of the image acquisition device; the field of view range of the image acquisition device covers all the goods stored on the warehouse position; positioning and detecting the warehouse position object in the warehouse position map; identifying the goods placement state of the warehouse position object to obtain the goods placement state of the warehouse position represented by the warehouse position object; the goods placement state is used to indicate whether there is goods on the warehouse position represented by the warehouse position object; controlling and processing the goods placement of the carrying device according to the goods placement state.

2. The method of claim 1, wherein, Before the step of acquiring the warehouse position map obtained by image acquisition, the method further comprises the step of installing the image acquisition device; the step of installing the image acquisition device comprises: calculating the field of view range of the image acquisition device according to the highest goods height, the device parameters of the image acquisition device and the preset installation position; if the goods on the target warehouse position can be completely displayed in the field of view range, installing the image acquisition device according to the preset installation position; wherein the target warehouse position is the warehouse position corresponding to the real world for storing goods represented by each warehouse position object in the warehouse position map.

3. The method of claim 1, wherein, The positioning and detecting the warehouse position object in the warehouse position map comprises: acquiring a pre-configuration file, and obtaining the warehouse position information of the warehouse position object from the pre-configuration file; the warehouse position information is obtained by calibrating a sample warehouse position map; the sample warehouse position map is obtained by pre-acquiring the image of the warehouse position represented by the warehouse position object by the image acquisition device; positioning and detecting the warehouse position map according to the warehouse position information to obtain the warehouse position object in the warehouse position map.

4. The method of claim 1, wherein, The goods placement state is obtained by identifying the goods placement state of the goods object by a goods state recognition engine; the goods state recognition engine is converted from a goods state recognition model trained by deep learning; the detection and inference speed of the goods state recognition engine is better than that of the goods state recognition model.

5. The method according to any one of claims 1 to 4, characterized in that, The controlling and processing the goods placement of the carrying device according to the goods placement state comprises: if the goods placement state is no goods state, generating a goods operation signal to make the carrying device place goods in the warehouse position represented by the warehouse position object according to the goods operation signal.

6. The method of claim 5, wherein, The method further comprises: if the goods placement state is the goods state, generating a notification for prompting that the warehouse position represented by the warehouse position object has goods.

7. The method of claim 5, wherein, The method further comprises: if the goods placement state of the warehouse position represented by a plurality of warehouse position objects is no goods state, After the handling device places goods on a last storage location in the empty state, a next goods operation signal is generated to control the handling device to place goods on a next storage location corresponding to the empty state based on the next goods operation signal.

8. The method of claim 1, wherein, The storage locations are arranged in a loading platform, and the image acquisition device is fixedly installed above the loading platform, and a field of view range of the image acquisition device covers goods stored on all the storage locations in the loading platform.

9. A cargo placement control apparatus characterized by comprising: The device comprises: An image acquisition module is configured to acquire a storage location image obtained by image acquisition of a storage location; the storage location image includes at least one storage location object; the storage location object represents a storage location for storing goods; the storage location image is acquired by an image acquisition device; the image acquisition device has a horizontal viewing angle and a vertical viewing angle at a preset installation height; a calculation result of the preset installation height and a highest goods height is a field of view length and a field of view width; a calculation result of the field of view length and the field of view width is a field of view range of the image acquisition device; and the field of view range of the image acquisition device covers goods stored on all the storage locations. A positioning detection module is configured to positionally detect the storage location object in the storage location image. A state recognition module is configured to recognize a goods placement state of the storage location object, to obtain a goods placement state of the storage location represented by the storage location object; and the goods placement state is used to indicate whether there is goods on the storage location represented by the storage location object. A placement control module is configured to perform goods placement control processing on a handling device according to the goods placement state. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

Images