Logistics operation automatic inspection method and system based on computer visual identification technology

Abstract

The invention discloses a logistics operation automatic inspection method based on a computer visual identification technology. The method comprises the steps that videos of logistics unit field operation are collected in real time through high-definition cameras in cabins, trucks, wharfs and warehouse loading and unloading areas; collecting videos of field operation of a logistics unit in real time according to a preset inspection route through an inspection robot in an indoor warehouse and an outdoor cargo storage yard; the method comprises the following steps: generating video streams from videos of a high-definition camera and an inspection robot through a video frame extraction server, and sequentially generating a video frame picture set according to a preset extraction speed; outputting a logistics operation violation behavior criterion through the trained computer vision large model; whether the picture content in the picture set has the logistics operation violation behavior or not is judged according to the logistics operation violation behavior criterion, if yes, specific violation content is marked on the picture with the violation behavior, and logistics management personnel in the enterprise can check the specific violation content online; and statistical analysis and visual large-screen display are carried out on the information of the violation content.

Description

Technical Field

[0001] The present invention relates to the field of automatic inspection systems for logistics operations, and in particular to an automatic inspection method and system for logistics operations based on computer vision recognition technology. Background Art

[0002] Steel products are bulk raw materials. After they are produced, they need to be transported by a third-party logistics company with professional qualifications to deliver the finished steel products to the delivery location designated by the steel companies and their downstream end users. Currently, from the time the finished products leave the factory to the time they are delivered to the user, they need to go through multiple logistics links such as factory shipment, port unloading, and warehousing. Illegal operations in any link may cause damage to the quality of steel products, thereby affecting delivery to users.

[0003] After analysis: the factory shipment link is concentrated in the steel enterprises' own finished product terminals, and the operation is relatively standardized; the port unloading link mainly utilizes social terminal resources and is managed by third-party logistics companies with uneven management levels. Although some terminals have installed video surveillance equipment, the cost of manual identification of abnormalities and violations is high; the warehousing link involves indoor warehouses and outdoor yards, which are widely distributed, and the goods are stacked at different heights and have a large storage area. The coverage area of ​​existing video surveillance equipment in warehouses is limited.

[0004] At present, steel enterprises adopt patrol inspection methods to check the operation standardization of many logistics nodes such as unloading and warehousing. There are two main methods: one is to arrange internal logistics management personnel to regularly inspect the logistics operation site to identify and correct abnormal and illegal behaviors in a timely manner; the other is to require logistics operation units to submit on-site monitoring videos regularly, and analyze whether various operations comply with the operation specifications through video images. For the first method, due to the limited energy of management personnel, it is impossible to achieve full coverage of operation time and location; for the second method, management personnel need to spend a lot of energy to play back the video collected in the video storage card through the video player, find out the places where there are violations and mark them with documents, and finally form a rectification form based on the document and submit it to the logistics operation unit for feedback and rectification. Therefore, the above methods add a lot of management costs to the enterprise, and cannot achieve timely discovery and effective containment of illegal operations. Summary of the invention

[0005] The purpose of the present invention is to provide an automatic inspection method and system for logistics operations based on computer vision recognition technology, which can realize real-time monitoring of the current operating status of each video point, so as to timely discover and effectively curb illegal operations and reduce the management cost of the enterprise.

[0006] The automatic inspection method for logistics operations based on computer vision recognition technology of the present invention comprises the following steps: S1. Use high-definition cameras to collect real-time videos of logistics units’ on-site operations in ship cabins, trucks, docks, and warehouse loading and unloading areas; S2, in indoor warehouses and outdoor cargo yards, patrol robots collect real-time videos of on-site operations of logistics units according to preset patrol routes; S3, using the video frame extraction server to generate a video stream from the video of the high-definition camera and the inspection robot, and sequentially generate a video frame picture set from the video stream according to a preset extraction speed; S4. Train a computer vision big model, and output the code of illegal behavior in logistics operations through the trained computer vision big model; S5, judging whether the image content in the image collection contains any violation of logistics operation rules through the inference server according to the violation rules of logistics operation, and if there is any violation, marking the specific violation content on the image with the violation; S6. Provide pictures with specific illegal contents for online viewing by logistics management personnel within the enterprise, and conduct statistical analysis and visual large-screen display of the illegal contents.

[0007] As a preferred solution of the present invention, it also includes storing pictures marked with specific illegal content.

[0008] As a preferred solution of the present invention, it also includes providing pictures with specific illegal contents for online viewing by third-party logistics units, and providing feedback on rectification measures to logistics management personnel within the enterprise.

[0009] As a preferred solution of the present invention, the preset extraction speed is an extraction speed of 1 to 5 frames per second.

[0010] The automatic inspection system for logistics operations based on computer vision recognition technology described in the present invention includes a high-definition camera, an inspection robot, a video frame extraction server, a large computer vision model, an inference server, and an internal enterprise information system; The high-definition cameras are installed in cabins, trucks, docks, and warehouse loading and unloading areas to collect real-time video of on-site operations of logistics units and transmit them to the video frame extraction server; The inspection robot is installed in indoor warehouses and outdoor cargo yards, and collects real-time video of on-site operations of logistics units according to preset inspection routes and transmits it to the video frame extraction server; The video frame extraction server generates a video stream from the video of the high-definition camera and the inspection robot, and sequentially generates a video frame picture set according to a preset extraction speed and transmits it to the inference server; The computer vision large model obtains a training picture set, performs training and outputs a code of conduct for illegal logistics operations, and transmits it to an inference server; The inference server determines whether the content of the pictures in the picture collection contains any violation of the logistics operation according to the violation criterion of the logistics operation. If there is any violation, the specific violation content is marked on the picture with the violation; The enterprise's internal information system will provide pictures with specific illegal content for online viewing by logistics management personnel within the enterprise, and will conduct statistical analysis and large-screen visualization of the information on the illegal content.

[0011] As a preferred solution of the present invention, an object storage server is also included. The object storage service is used to store pictures marked with specific illegal content and make them available for the internal information system of the enterprise.

[0012] As a preferred solution of the present invention, it also includes an enterprise 3PL management system, which is used to call the object storage server to store pictures with specific illegal content marked for online viewing by third-party logistics units, and to feedback corrective measures to the enterprise's internal information system.

[0013] As a preferred solution of the present invention, the high-definition camera and the inspection robot are connected to the video frame extraction server signal through a 4G or 5G network.

[0014] The automatic inspection method and system for logistics operations based on computer vision recognition technology described in the present invention generates a video stream by using the video of a high-definition camera and an inspection robot, and sequentially generates a video frame picture set according to a preset extraction speed. The computer vision large model is then trained to output a criterion for logistics operation violations to determine whether the picture content in the picture set contains logistics operation violations. The specific violation content is marked on the pictures with violations for online viewing by logistics management personnel within the enterprise, and the information on the violation content is statistically analyzed and visualized on a large screen, thereby achieving real-time monitoring of the current operation status of each video point, and timely discovery and prevention of illegal operations. Effectively curb and reduce the management cost of the enterprise; in addition, high-definition cameras are installed in cabins, trucks, docks, and warehouse loading and unloading areas, and inspection robots are set up in indoor warehouses and outdoor cargo yards. According to the preset inspection routes, real-time video of on-site operations of logistics units can be collected, so that the current operation status of each video point can be fully covered and monitored in real time, and then violations of logistics operations can be discovered in time; in addition, the trained computer vision large model outputs the logistics operation violation criteria to determine whether there are logistics operation violations in the picture content of the picture set, so that the judgment is more accurate, and further illegal operations can be discovered and effectively curbed in time, reducing the management cost of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a block diagram of the automatic inspection system for logistics operations based on computer vision recognition technology of the present invention; Figure 2It is a flow chart of the automatic inspection method of logistics operations based on computer vision recognition technology of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, an automatic inspection system for logistics operations based on computer vision recognition technology includes a high-definition camera, an inspection robot, a video frame extraction server, a computer vision large model, an inference server, an enterprise internal information system, an object storage server, and an enterprise 3PL management system.

[0018] HD cameras are installed in cabins, trucks, docks, warehouse loading and unloading areas, and collect real-time video of on-site operations of logistics units and transmit it to the video frame extraction server through 4G or 5G network. HD cameras can be installed in cabins, trucks, docks, warehouses and other loading and unloading areas by using tripods, wall mounts, magnetic suction, etc.

[0019] Inspection robots are installed in indoor warehouses and outdoor cargo yards. They collect real-time video of on-site operations of logistics units according to preset inspection routes and transmit them to video frame extraction servers via 4G or 5G networks. Fixed inspection routes can be set according to the environmental characteristics of the inspection area and inspection requirements to ensure full coverage of the inspection targets and scopes.

[0020] The video frame extraction server generates video streams from the videos of high-definition cameras and inspection robots, and sequentially generates video frame picture sets according to the preset extraction speed and transmits them to the inference server. It can generate video frame picture sets at a speed of 1 to 5 frames per second, thereby fully covering the real-time monitoring of the current operating status of each video point, and promptly discovering violations in logistics operations.

[0021] The computer vision large model obtains the training picture set, performs training and outputs the logistics operation violation behavior criteria, which are transmitted to the inference server.

[0022] The inference server determines whether the image content in the image set contains any logistics operation violations based on the logistics operation violation criteria. If there are any violations, the specific violation content will be marked on the image where the violation occurs, and the image marked with the specific violation content will be transmitted to the object storage server for storage.

[0023] The internal information system of the enterprise retrieves the images with specific illegal contents marked on the object storage server for online viewing by the logistics management personnel within the enterprise, and performs statistical analysis and large-screen visualization of the illegal contents.

[0024] The enterprise's 3PL management system retrieves images with specific illegal content marked on the object storage server for online viewing by third-party logistics units, and feedbacks corrective measures to the enterprise's internal information system.

[0025] An automatic inspection method for logistics operations based on computer vision recognition technology, such as Figure 2 As shown, the following steps are included: S1. Use high-definition cameras to collect real-time video of logistics units’ on-site operations in ship cabins, trucks, docks, and warehouse loading and unloading areas.

[0026] S2. In indoor warehouses and outdoor cargo yards, patrol robots collect real-time videos of on-site operations of logistics units based on preset patrol routes.

[0027] S3. The video of the high-definition camera and the inspection robot is used to generate a video stream through the video frame extraction server, and the video stream is sequentially generated into a video frame picture set according to the preset extraction speed. The video frame picture set can be generated at a speed of 1 to 5 frames per second, so that the current operation status of each video point can be fully covered and monitored in real time, thereby promptly discovering violations in logistics operations.

[0028] S4. Train a large computer vision model and output the code of violations in logistics operations through the trained large computer vision model.

[0029] S5. The inference server determines whether the image content in the image collection contains any violation of the logistics operation according to the violation rules of the logistics operation. If there is any violation, the specific violation content is marked on the image with the violation. The images with the specific violation content are also stored.

[0030] S6. The pictures with specific illegal contents will be provided for online viewing by the logistics management personnel within the enterprise, and the information of illegal contents will be statistically analyzed and displayed on a large screen. At the same time, the pictures with specific illegal contents will be provided for online viewing by third-party logistics units, and the rectification measures will be fed back to the logistics management personnel within the enterprise. The logistics management personnel of the enterprise can require the logistics operation unit to take intervention measures as soon as possible, verify the on-site operation situation, supervise the rectification and feedback the results, so as to minimize the risk of logistics damage. The logistics management personnel can also confirm the feedback results of the logistics operation unit, and move the management from offline to online, monitor the abnormality of logistics operation in real time, effectively reduce illegal operations, and build an efficient, standardized and transparent quality management monitoring and improvement mechanism.

[0031] The above embodiments are only used to illustrate the detailed scheme of the present invention, and the present invention is not limited to the above detailed scheme, that is, it does not mean that the present invention must rely on the above detailed scheme to be implemented. Those skilled in the art should understand that any improvement of the present invention, equivalent replacement of the raw materials of the product of the present invention, addition of auxiliary components, selection of specific methods, etc., are all within the protection scope and disclosure scope of the present invention.

Claims

1. An automatic inspection method for logistics operations based on computer vision recognition technology, characterized in that: include: S1. Use high-definition cameras to collect real-time videos of logistics units’ on-site operations in ship cabins, trucks, docks, and warehouse loading and unloading areas; S2, in indoor warehouses and outdoor cargo yards, patrol robots collect real-time videos of on-site operations of logistics units according to preset patrol routes; S3, using the video frame extraction server to generate a video stream from the video of the high-definition camera and the inspection robot, and sequentially generate a video frame picture set from the video stream according to a preset extraction speed; S4. Train a computer vision big model, and output the code of illegal behavior in logistics operations through the trained computer vision big model; S5, judging whether the image content in the image collection contains any violation of logistics operation rules through the inference server according to the violation rules of logistics operation, and if there is any violation, marking the specific violation content on the image with the violation; S6. Provide pictures with specific illegal contents for online viewing by logistics management personnel within the enterprise, and conduct statistical analysis and large-screen visualization of the illegal contents.

2. The automatic inspection method for logistics operations based on computer vision recognition technology according to claim 1 is characterized in that: It also includes storing pictures with specific illegal content marked on them.

3. The automatic inspection method for logistics operations based on computer vision recognition technology according to claim 1 is characterized in that: It also includes making pictures with specific violations marked for online viewing by third-party logistics units, and providing feedback on corrective measures to the company's internal logistics management personnel.

4. The automatic inspection method for logistics operations based on computer vision recognition technology according to claim 1 is characterized in that: The preset extraction speed is an extraction speed of 1 to 5 frames per second.

5. An automatic inspection system for logistics operations based on computer vision recognition technology, characterized in that: Including high-definition cameras, inspection robots, video frame extraction servers, large computer vision models, inference servers, and internal enterprise information systems; The high-definition cameras are installed in cabins, trucks, docks, and warehouse loading and unloading areas to collect real-time video of on-site operations of logistics units and transmit them to the video frame extraction server; The inspection robot is installed in indoor warehouses and outdoor cargo yards, and collects real-time video of on-site operations of logistics units according to preset inspection routes and transmits it to the video frame extraction server; The video frame extraction server generates a video stream from the video of the high-definition camera and the inspection robot, and sequentially generates a video frame picture set according to a preset extraction speed and transmits it to the inference server; The computer vision large model obtains the training picture set, performs training and outputs the logistics operation violation behavior criteria and transmits them to the inference server; The inference server determines whether the images in the image collection contain any violations of logistics operations according to the logistics operation violation criteria. If there are any violations, the specific violations are marked on the images with the violations. The company’s internal information system will provide pictures with specific illegal content for online viewing by logistics management personnel within the company, and will conduct statistical analysis and large-screen visualization of the information on illegal content.

6. The automatic inspection system for logistics operations based on computer vision recognition technology according to claim 5 is characterized in that: It also includes an object storage server, which is used to store images marked with specific illegal content and make them available for the company's internal information system.

7. The automatic inspection system for logistics operations based on computer vision recognition technology according to claim 6 is characterized in that: It also includes the enterprise 3PL management system, which is used to call the object storage service to store pictures with specific illegal content marked for online viewing by third-party logistics units, and to feedback corrective measures to the enterprise's internal information system.

8. The automatic inspection system for logistics operations based on computer vision recognition technology according to claim 5 is characterized in that: High-definition cameras and inspection robots are connected to the video frame extraction server signals via 4G or 5G networks.

Images