A logistics inspection method and system based on machine vision

Abstract

The present invention proposes a logistics inspection method and system based on machine vision, which relates to the field of machine vision technology. Based on the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device, a logistics inspection queue and coding of multiple inspection mobile devices are established, and real-time target locking and distance keeping are performed based on a target locking algorithm and a distance keeping algorithm. The inspection mobile devices patrolling the logistics inspection queue obtain images of the surrounding environment in real time through visual sensors, detect and locate foreign objects in each image, identify obstacles on the image, map the diameter of the identified obstacle with different preset obstacle diameter intervals, and select a corresponding deflection angle to bypass the obstacle according to the mapping result. If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter interval, the inspection mobile device cannot pass, and the logistics inspection queue and coding need to be adjusted.

Description

Technical Field

[0001] The invention proposes a logistics inspection method and system based on machine vision, and relates to the technical field of machine vision. Background Art

[0002] With the development of the logistics industry, the scale of logistics warehouses is getting larger and larger; intelligent control technologies, such as the intelligent inspection system based on 5G technology, take advantage of its high speed and low latency to realize real-time monitoring and remote control of logistics equipment.

[0003] The application of mobile Internet technology enables inspection data to be uploaded and accessed in real time, improving the flexibility and efficiency of inspection work. The combination and application of these technologies make logistics inspection more intelligent and automated, improving the overall efficiency and safety of the logistics system. The application of Internet of Things technology in the field of logistics has realized the monitoring of the entire logistics warehouse and improved the efficiency and safety of logistics goods storage. The intelligent inspection system uses sensors, big data analysis and artificial intelligence algorithms to achieve real-time monitoring of goods and fault warning, thereby improving operation and maintenance efficiency and reducing costs.

[0004] The single inspection method of the existing technology is no longer suitable for the responsible warehouse structure and cannot adapt to the effective control of obstacle handling in cargo storage. The performance and functions of the inspection mobile device are also urgently needed to be improved to adapt to wider applications and growing market demand. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a logistics inspection method based on machine vision, comprising the following steps:

[0006] S1. Establishing a logistics inspection queue and coding of multiple inspection mobile devices based on the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device;

[0007] S2, each accompanying patrol mobile device obtains a field of view image including the previous patrol mobile device through a visual sensor, and performs real-time target locking and distance keeping based on a target locking algorithm and a distance keeping algorithm;

[0008] S3. The inspection mobile device that patrols the logistics inspection queue obtains images of the surrounding environment in real time through visual sensors, detects and locates foreign objects in each image, and identifies obstacles on the image;

[0009] S4, mapping the diameter of the identified obstacle with different preset obstacle diameter intervals, and selecting a corresponding deflection angle to bypass the obstacle according to the mapping result;

[0010] S5. If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the inspection mobile device cannot pass through, and the logistics inspection queue and coding need to be adjusted.

[0011] Furthermore, in step S2, the target locking algorithm calculates the coordinates of the center point of the rectangular area where the target to be followed is located in the field of view image and the center area of ​​the field of view image, and always keeps the coordinates of the center point of the rectangular area located in the center area of ​​the field of view image;

[0012] The distance keeping algorithm calculates the difference between the real-time distance between the patrol mobile device and the target and the optimal distance, and controls the difference to be kept within the optimal difference interval to achieve distance keeping.

[0013] Furthermore, suppose G X G is the lateral distance between the patrol mobile device and the target, Y is the longitudinal distance between the patrol mobile device and the target, and the optimal transverse distance between the patrol mobile device and the target is G X0 , the optimal longitudinal distance is G Y0 , calculate the horizontal difference value and the vertical difference is :

[0014] ;

[0015] ;

[0016] Set the horizontal optimal difference interval to , set the longitudinal optimal difference interval to , control the horizontal difference value and vertical difference The values ​​are kept within the optimal horizontal difference range and the optimal vertical difference range respectively to achieve distance maintenance.

[0017] Furthermore, a deep learning model is used to detect and locate foreign objects in the image and identify obstacles on the image;

[0018] Define the recognition function L of the deep learning model:

[0019] ;

[0020] in, The Tth foreign object recognition bounding box, The obstacle template bounding box corresponding to the Tth foreign object recognition bounding box, It is a foreign object recognition bounding box The center point and obstacle template bounding box The difference distance between the center points of and The diagonal length of the minimum enclosing area of ​​; is the category weight, N is the total number of foreign body recognition bounding boxes, T is the index of the foreign body recognition bounding box, is the overlap ratio function;

[0021] When the output value of the recognition function L is less than the difference threshold, it means that there is an obstacle on the image, and an obstacle warning and obstacle position reporting are performed.

[0022] Furthermore, the overlap ratio function is:

[0023] ;

[0024] in, Identify bounding boxes for foreign objects Bounding box with obstacle template The overlapping area, Identify bounding boxes for foreign objects Bounding box with obstacle template The union area of ​​.

[0025] Furthermore, in step S4, different diameter intervals of obstacles are pre-set, each diameter interval corresponds to a steering angle, and the diameter interval of the obstacle mapped by the diameter of the identified obstacle is determined, and the corresponding steering angle is determined according to the diameter interval mapping result;

[0026] The leading inspection mobile device leads the accompanying inspection mobile devices to circumvent obstacles in sequence according to the deflection angles.

[0027] Furthermore, in step S5, if the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the leading inspection mobile device cannot bypass the obstacle, and a new leading inspection mobile device needs to be selected;

[0028] Analyze the following position coordinates of multiple following inspection mobile devices, select the following inspection mobile device located at the fork intersection as the new leading inspection mobile device, and re-establish the logistics inspection queue and codes of the multiple inspection mobile devices according to step 1 to obtain a new logistics inspection queue and a new code.

[0029] Furthermore, let the path evaluation coefficient J of the qth fork be q The calculation expression is:

[0030] ;

[0031] Where, v is the speed of the inspection mobile device, Z q is the path selection index obtained from historical big data statistics, K qThe probability of obstacles appearing on the path obtained from historical big data statistics, is the maximum length of the path, is the differential length of the path, is the adjustment factor;

[0032] The following inspection mobile device corresponding to the fork in the road with the smallest path evaluation coefficient is used as the new leading inspection mobile device.

[0033] The present invention also proposes a logistics inspection system based on machine vision, which is used to implement the above-mentioned logistics inspection method based on machine vision. The logistics inspection system includes: multiple inspection mobile devices, wireless positioning devices, queue scheduling subsystems, multiple tracking processing subsystems, multiple image processing subsystems and multiple obstacle processing units;

[0034] The multiple patrol inspection mobile devices include a leading patrol inspection mobile device and multiple accompanying patrol inspection mobile devices; each patrol inspection mobile device is provided with a visual sensor for acquiring an image of the field of view of the previous patrol inspection mobile device and an image of the surrounding environment;

[0035] The wireless positioning device uses wireless positioning technology to determine the precise position coordinates of each patrol mobile device by measuring the distance difference between the patrol mobile device and multiple base stations to form the leading position coordinates and the following position coordinates;

[0036] The queue arrangement subsystem calculates the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device based on the leading position coordinates and the following position coordinates, and establishes a logistics inspection queue and coding for multiple inspection mobile devices;

[0037] Each patrol mobile device is equipped with a tracking processing subsystem, an image processing subsystem and an obstacle processing unit. The tracking processing subsystem uses a target locking algorithm and a distance keeping algorithm to perform real-time target locking and distance keeping along with the patrol mobile device; the image processing subsystem uses a visual sensor to obtain images of the surrounding environment in real time, uses a deep learning model to detect and locate foreign objects in each image, and identifies obstacles on the image; the obstacle processing unit maps the diameter of the obstacle with different preset obstacle diameter intervals, and selects the corresponding deflection angle to bypass the obstacle based on the mapping result.

[0038] Further, the tracking processing subsystem includes a target locking module and a distance keeping module;

[0039] The target locking module calculates the coordinates of the center point of the rectangular area where the target to be followed is located in the field of view image and the center area of ​​the field of view image, and always keeps the coordinates of the center point of the rectangular area located in the center area of ​​the field of view image;

[0040] The distance keeping module tracks the distance to the target through a distance keeping algorithm, calculates the difference between the real-time distance between the patrol mobile device and the target and the set optimal distance, sets the optimal difference interval, controls the difference value to remain within the optimal difference interval, and realizes distance keeping.

[0041] Compared with the prior art, the present invention has the following beneficial technical effects:

[0042] (1) With the development of the logistics industry, the scale of logistics warehouses is getting larger and larger. The single inspection method of the existing technology is no longer suitable for the warehouse structure. Therefore, the present invention establishes inspection queues and codes of multiple inspection mobile devices. Through the coordinated work of multiple inspection mobile devices, a larger inspection range can be covered, reducing the time required for a single inspection.

[0043] (2) Since multiple patrol mobile devices are performing logistics inspections at the same time, an algorithm needs to be designed to ensure that they can follow each other without collision. Specifically, each accompanying patrol mobile device performs real-time target locking and distance maintenance through a target locking algorithm and a distance maintenance algorithm. Through the precise target locking algorithm, the patrol mobile device can identify and track the target in real time, ensuring the continuity and efficiency of the inspection work. At the same time, the distance maintenance algorithm can prevent collisions between patrol mobile devices and ensure the safety of the inspection process.

[0044] (3) The patrol mobile device that patrols according to the patrol queue uses visual sensors to obtain real-time images of the surrounding environment, detects and locates foreign objects in each image, and identifies obstacles in the image. Combined with advanced technologies such as deep learning, the patrol mobile device can not only detect obstacles, but also perform more complex scene understanding.

[0045] (4) Map the obstacle diameter with different preset obstacle diameter intervals, and select the corresponding deflection angle to bypass the obstacle based on the mapping result; if the obstacle diameter exceeds the maximum value of the preset obstacle diameter interval, the inspection mobile device cannot pass through, and the inspection queue and encoding need to be adjusted. By matching the actual size of the obstacle with the preset size interval, the inspection mobile device can adopt different obstacle avoidance strategies for obstacles of different sizes, thereby improving the adaptability and flexibility of the obstacle avoidance behavior; reducing unnecessary detours, optimizing the overall inspection path, and improving inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0047] Figure 1 It is a flow chart of the logistics inspection method based on machine vision of the present invention;

[0048] Figure 2 A schematic diagram of maintaining the distance between the accompanying inspection mobile device and the target of the present invention;

[0049] Figure 3 It is a schematic diagram of a mobile device for following inspections located at a forked road intersection according to the present invention;

[0050] Figure 4 It is a structural schematic diagram of the logistics inspection system based on machine vision of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] In the drawings of the specific embodiments of the present invention, in order to better and more clearly describe the working principles of the various components in the system, the connection relationship of the various parts in the device is shown, which only clearly distinguishes the relative position relationship between the various components, and cannot constitute a limitation on the signal transmission direction, connection sequence and structural size, size and shape of each part within the component or structure.

[0053] Example 1

[0054] This embodiment proposes a logistics inspection method based on machine vision, such as Figure 1 As shown, the logistics inspection method includes the following steps:

[0055] S1. Establish inspection queues and codes for multiple inspection mobile devices based on the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device.

[0056] Before establishing a queue of multiple patrol mobile devices, two key coordinate points must be set first: the leading position coordinate and the following position coordinate. The leading position coordinate is the position coordinate of the leading patrol mobile device, while the following position coordinate is the position coordinate of multiple accompanying patrol mobile devices.

[0057] The leading inspection mobile device is the first mobile device to scan the inspection site, and is used to assist the accompanying inspection mobile device in route arrangement. The distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device is calculated, and the accompanying inspection mobile device closest to the leading inspection mobile device is obtained, and the closest accompanying inspection mobile device is set as the first accompanying inspection mobile device, and the accompanying inspection mobile device closest to the first accompanying inspection mobile device is set as the second accompanying inspection mobile device, and so on, and the serial numbers of the remaining multiple accompanying inspection mobile devices are set, and the inspection queue encoding is completed.

[0058] In a preferred embodiment, the Euclidean distance formula is used to calculate the distance between the accompanying inspection mobile device and the leading inspection mobile device. The calculation formula of the distance D is as follows:

[0059] ;

[0060] Among them, (x1, y1) is the position coordinate of the leading inspection mobile device, and (x2, y2) is the position coordinate of the accompanying inspection mobile device.

[0061] For each accompanying inspection mobile device, use the above formula to calculate the distance between it and the leading inspection mobile device, compare all calculated distances, find the smallest distance, and the corresponding accompanying inspection mobile device is the first accompanying inspection mobile device.

[0062] From the remaining accompanying inspection mobile devices, find out the one with the shortest distance to the first accompanying inspection mobile device and set it as the second accompanying inspection mobile device.

[0063] Repeat this process until all accompanying inspection mobile devices have serial numbers.

[0064] Preferably, the wireless positioning technology is used to determine the precise position coordinates of each patrol mobile device by measuring the distance difference between the patrol mobile device and multiple base stations. The position coordinates of the patrol mobile device are updated in real time, so that the patrol mobile device can dynamically determine its own position during movement.

[0065] S2. Each accompanying inspection mobile device performs real-time target locking and distance keeping through a target locking algorithm and a distance keeping algorithm.

[0066] Each accompanying inspection mobile device obtains the field of view image including the previous inspection mobile device through the visual sensor, determines the target to be followed, selects the rectangular area where the target to be followed is located according to the position of the target to be followed in the field of view image, and performs real-time target position tracking through the target locking algorithm.

[0067] The target locking algorithm calculates the coordinates of the center point of the rectangular area where the target to be followed is located in the field of view image and the center area of ​​the field of view image, and always keeps the coordinates of the center point of the rectangular area in the center area of ​​the field of view image. Specifically:

[0068] First, determine the coordinates of the center point of the rectangular area:

[0069] Assume the coordinates of the upper left corner of the rectangular area are (A1, B1), and the coordinates of the lower right corner are (A2, B2).

[0070] The center point coordinates (A0, B0) can be calculated using the following formula:

[0071] A0=(A1+A2) / 2;

[0072] B0=(B1+B2) / 2;

[0073] Then, determine the central area of ​​the field of view image:

[0074] The central area of ​​the field of view image refers to an area around the center point of the field of view image. Assuming the width of the field of view image is W and the height is H, then the coordinates of the center point (V1, V2) are:

[0075] V1=W / 2;

[0076] V2=H / 2;

[0077] If the center point of the rectangular area is not within the center area of ​​the field of view image, you need to adjust the position of the center area of ​​the field of view image, that is, adjust the posture of the inspection mobile device. This can be achieved by calculating the deviation between the center point of the rectangular area and the center point of the center area of ​​the field of view image, and moving the field of view image accordingly.

[0078] In a preferred embodiment, range-keeping tracking to a target location is performed by a range-keeping algorithm.

[0079] like Figure 2 As shown, initially the accompanying inspection mobile device is located at point Q1, rectangle 10 is the field of view image of the accompanying inspection mobile device at the current moment, assuming that the target to be followed is O, that is, the previous inspection mobile device, and assuming that the accompanying inspection mobile device is located at point Q2 at the next moment, and the field of view image is 20.

[0080] G XG is the lateral distance between the patrol mobile device and the target, Y is the longitudinal distance between the accompanying inspection mobile device and the target. In order to avoid collision between the accompanying inspection mobile device and the target during the following process, the optimal following position is given. When determining the optimal position, the optimal lateral distance G between the accompanying inspection mobile device and the target is set. X0 and the optimal longitudinal distance G Y0 .

[0081] Calculate the difference between the real-time distance and the optimal distance, and set the horizontal difference value to , the vertical difference value is :

[0082] ;

[0083] ;

[0084] Set the horizontal optimal difference interval to , set the longitudinal optimal difference interval to , control the horizontal difference value and vertical difference The values ​​are kept within the optimal horizontal difference range and the optimal vertical difference range respectively to achieve distance maintenance.

[0085] S3. The mobile inspection device patrolling along the inspection queue acquires images of the surrounding environment in real time through visual sensors, detects and locates foreign objects in each image, and identifies obstacles on the image.

[0086] First, each patrol mobile device captures images of the surrounding environment through visual sensors. Through image information, the patrol mobile device can understand key elements such as obstacles, road conditions, navigation signs, etc. on the patrol path in real time.

[0087] In a preferred embodiment, the image of the surrounding environment captured by the visual sensor is enhanced, for example, by using a stitching method of random scaling, cropping and translation. The above method is suitable for small-sized objects, can make the objects clearer, and can also reduce the amount of calculation.

[0088] In a preferred embodiment, a deep learning model is used to detect and locate foreign objects in an image and identify obstacles on the image. The deep learning model uses a recognition function to measure the difference between the foreign object recognition result and the obstacle template.

[0089] Specifically, define the recognition function L of the deep learning model:

[0090] ;

[0091] in, The Tth foreign object recognition bounding box, The obstacle template bounding box corresponding to the Tth foreign object recognition bounding box, It is a foreign object recognition bounding box The center point and obstacle template bounding box The difference distance between the center points of and The diagonal length of the minimum enclosing area of ​​; is the category weight, N is the total number of foreign object recognition bounding boxes, and T is the index of the foreign object recognition bounding box; is the overlap ratio function, and the calculation formula is:

[0092] ;

[0093] in, Identify bounding boxes for foreign objects Bounding box with obstacle template The overlapping area, Identify bounding boxes for foreign objects Bounding box with obstacle template The union area of ​​.

[0094] Finally, when the output value of the recognition function L is less than the difference threshold, it means that there is an obstacle on the image, and an obstacle warning and obstacle position reporting are performed.

[0095] S4. Mapping is performed according to the diameter of the obstacle and different preset obstacle diameter intervals, and a corresponding deflection angle is selected according to the mapping result to bypass the obstacle.

[0096] When an obstacle is identified on the image through step S3, the diameter L of the obstacle is obtained from the obstacle data, and the obstacle diameter L is mapped with the preset obstacle diameter range, and the corresponding deflection angle is selected as the turning angle of the inspection mobile device for avoidance according to the mapping result.

[0097] Specifically, different diameter intervals of obstacles are pre-set, each diameter interval corresponds to a steering angle, and the diameter L of the identified obstacle belongs to which diameter interval. Each diameter interval of the obstacle corresponds to a deflection angle. For example, if the diameter value in the diameter interval is long, the deflection angle of the corresponding inspection mobile device also needs to be increased. According to the diameter interval mapping result, the corresponding steering angle is determined.

[0098] The leading inspection mobile device leads the accompanying inspection mobile devices to select corresponding deflection angles according to the mapping results to circumvent obstacles in turn.

[0099] S5. If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the inspection mobile device cannot pass through, and the inspection queue and coding need to be adjusted.

[0100] If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the leading inspection mobile device cannot bypass the obstacle, and it is necessary to abandon the current leading inspection mobile device and wait for the obstacle to be processed.

[0101] At the same time, based on the map data, the following position coordinates of multiple following inspection mobile devices are analyzed to determine which following inspection mobile device is located at the fork in the road, and the sequence number of the following inspection mobile device located at the fork in the road is selected, for example Figure 3 The number 3 follow-up inspection mobile device is shown.

[0102] The following inspection mobile device is switched to the leading inspection mobile device, and the inspection queues and codes of the multiple inspection mobile devices are re-established according to step 1 to obtain a new inspection queue and a new code.

[0103] After switching to a new leading inspection mobile device, the new leading inspection mobile device selects an alternative inspection route for inspection.

[0104] In a preferred embodiment, a new lead inspection mobile device is selected based on the path evaluation coefficient.

[0105] Specifically, the path evaluation coefficient J of the qth fork is q The calculation expression is:

[0106] ;

[0107] Where, v is the speed of the inspection mobile device, Z q The selection index of the path is obtained by statistics of historical big data, K q is the probability of obstacles appearing on the path obtained from historical big data statistics, is the maximum length of the path, is the differential length of the path, is the adjustment factor.

[0108] The following inspection mobile device corresponding to the fork in the road with the smallest path evaluation coefficient is used as the new leading inspection mobile device.

[0109] Example 2

[0110] This embodiment proposes a logistics inspection system method based on machine vision, such as Figure 4 As shown, the logistics inspection system includes: multiple inspection mobile devices 100, a wireless positioning device 200, a queue scheduling subsystem 300, a tracking processing subsystem 400, an image processing subsystem 500, and an obstacle processing unit 600. Figure 4 The figure only shows, by way of example, a built-in tracking processing subsystem, an image processing subsystem and an obstacle processing unit of a patrol mobile device.

[0111] A plurality of patrol mobile devices include a leading patrol mobile device and a plurality of accompanying patrol mobile devices.

[0112] Each patrol mobile device is equipped with a visual sensor. Each accompanying patrol mobile device obtains the field of view image of the previous patrol mobile device through the visual sensor, and at the same time obtains the image of the surrounding environment in real time through the visual sensor, so as to understand the key elements such as obstacles, road conditions, navigation signs and so on on the patrol path in real time.

[0113] The wireless positioning device uses wireless positioning technology to determine the precise position coordinates of each patrol mobile device by measuring the distance difference between the patrol mobile device and multiple base stations, forming the leading position coordinates and the following position coordinates.

[0114] The queue scheduling subsystem is used to establish inspection queues and codes for multiple inspection mobile devices, calculate the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device, obtain the accompanying inspection mobile device closest to the leading inspection mobile device, and set the closest accompanying inspection mobile device as the first accompanying inspection mobile device, set the accompanying inspection mobile device closest to the first accompanying inspection mobile device as the second accompanying inspection mobile device, and so on, set the serial numbers of the remaining multiple accompanying inspection mobile devices, and then the inspection queue coding is completed.

[0115] Each patrol mobile device is built with a tracking processing subsystem, which performs real-time target locking and distance keeping in the patrol mobile device through a target locking algorithm and a distance keeping algorithm.

[0116] In a preferred embodiment, the tracking processing subsystem includes a target locking module and a distance keeping module.

[0117] The target locking module calculates the center point coordinates of the rectangular area where the target to be followed in the field of view image is located and the center area of ​​the field of view image, and always keeps the center point coordinates of the rectangular area located in the center area of ​​the field of view image.

[0118] The distance keeping module uses the distance keeping algorithm to keep tracking the distance with the target position, calculates the difference between the real-time distance and the optimal distance, sets the optimal difference interval, and controls the difference value to remain within the optimal difference interval to achieve distance keeping.

[0119] Each mobile inspection device has a built-in image processing subsystem and obstacle processing unit. The image processing subsystem acquires images of the surrounding environment in real time through visual sensors, uses a deep learning model to detect and locate foreign objects in each image, and identifies obstacles in the image.

[0120] The obstacle processing unit maps the diameter of the obstacle with different preset obstacle diameter intervals, and selects a corresponding deflection angle to bypass the obstacle according to the mapping result.

[0121] If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the obstacle processing unit determines that the inspection mobile device cannot pass through, and notifies the queue scheduling subsystem to adjust the inspection queue and code to obtain a new inspection queue and a new code.

[0122] The visual sensor-based logistics inspection method of the present invention not only improves the autonomy and flexibility of the inspection mobile device, but also greatly reduces the dependence on the external environment. Even in a complex and changeable environment, as long as the navigation signs can be identified, the mobile device can smoothly locate and navigate.

[0123] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0124] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0125] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A logistics inspection method based on machine vision, characterized in that: The steps include: S1. Establishing a logistics inspection queue and coding of multiple inspection mobile devices based on the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device; S2, each accompanying patrol mobile device obtains a field of view image including the previous patrol mobile device through a visual sensor, and performs real-time target locking and distance keeping based on a target locking algorithm and a distance keeping algorithm; S3. The inspection mobile device that patrols the logistics inspection queue obtains images of the surrounding environment in real time through visual sensors, detects and locates foreign objects in each image, and identifies obstacles on the image; S4, mapping the diameter of the identified obstacle with different preset obstacle diameter intervals, and selecting a corresponding deflection angle to bypass the obstacle according to the mapping result; S5. If the diameter of the obstacle exceeds the maximum value of the preset obstacle diameter range, the inspection mobile device cannot pass through, and the logistics inspection queue and code need to be adjusted, and the leading inspection mobile device needs to be reselected; Analyze the following position coordinates of multiple following inspection mobile devices, select the following inspection mobile device located at the fork intersection as the new leading inspection mobile device, and re-establish the logistics inspection queue and codes of the multiple inspection mobile devices according to step 1 to obtain a new logistics inspection queue and a new code; Assume the path evaluation coefficient J of the qth fork in the road q The calculation expression is: ； Where, v is the speed of the inspection mobile device, Z q is the path selection index obtained from historical big data statistics, K q The probability of obstacles appearing on the path obtained from historical big data statistics, is the maximum length of the path, is the differential length of the path, is the adjustment factor; The following inspection mobile device corresponding to the fork in the road with the smallest path evaluation coefficient is used as the new leading inspection mobile device.

2. The method for logistics inspection based on machine vision according to claim 1, characterized in that: In step S2, the target locking algorithm calculates the coordinates of the center point of the rectangular area where the target to be followed is located in the field of view image and the center area of ​​the field of view image, and always keeps the coordinates of the center point of the rectangular area located in the center area of ​​the field of view image; The distance keeping algorithm calculates the difference between the real-time distance between the patrol mobile device and the target and the optimal distance, and controls the difference to be kept within the optimal difference interval to achieve distance keeping.

3. The method for logistics inspection based on machine vision according to claim 2 is characterized in that: Let G X G is the lateral distance between the patrol mobile device and the target, Y is the longitudinal distance between the patrol mobile device and the target, and the optimal transverse distance between the patrol mobile device and the target is G X0 , the optimal longitudinal distance is G Y0 , calculate the horizontal difference value and the vertical difference is : ； ； Set the horizontal optimal difference interval to , set the longitudinal optimal difference interval to , control the horizontal difference value and vertical difference The values ​​are kept within the optimal horizontal difference range and the optimal vertical difference range respectively to achieve distance maintenance.

4. The method for logistics inspection based on machine vision according to claim 1, characterized in that: Use deep learning models to detect and locate foreign objects in images and identify obstacles in images; Define the recognition function L of the deep learning model: ； in, The Tth foreign object recognition bounding box, The obstacle template bounding box corresponding to the Tth foreign object recognition bounding box, It is a foreign object recognition bounding box The center point and obstacle template bounding box The difference distance between the center points of and The diagonal length of the minimum enclosing area of ​​; is the category weight, N is the total number of foreign body recognition bounding boxes, T is the index of the foreign body recognition bounding box, is the overlap ratio function; When the output value of the recognition function L is less than the difference threshold, it means that there is an obstacle on the image, and an obstacle warning and obstacle position reporting are performed.

5. The method for logistics inspection based on machine vision according to claim 4, characterized in that: The overlap ratio function is: ； in, Identify bounding boxes for foreign objects Bounding box with obstacle template The overlapping area, Identify bounding boxes for foreign objects Bounding box with obstacle template The union area of ​​.

6. The method for logistics inspection based on machine vision according to claim 1, characterized in that: In step S4, different diameter intervals of obstacles are pre-set, each diameter interval corresponds to a steering angle, the diameter interval of the obstacle mapped by the identified obstacle diameter is determined, and the corresponding steering angle is determined according to the diameter interval mapping result; The leading inspection mobile device leads the accompanying inspection mobile devices to circumvent obstacles in sequence according to the deflection angles.

7. A logistics inspection system based on machine vision, characterized in that: Used to implement the machine vision-based logistics inspection method as described in any one of claims 1 to 6, the logistics inspection system comprises: a plurality of inspection mobile devices, a wireless positioning device, a queue scheduling subsystem, a plurality of tracking processing subsystems, a plurality of image processing subsystems and a plurality of obstacle processing units; The multiple patrol inspection mobile devices include a leading patrol inspection mobile device and multiple accompanying patrol inspection mobile devices; each patrol inspection mobile device is provided with a visual sensor for acquiring an image of the field of view of the previous patrol inspection mobile device and an image of the surrounding environment; The wireless positioning device uses wireless positioning technology to determine the precise position coordinates of each patrol mobile device by measuring the distance difference between the patrol mobile device and multiple base stations to form the leading position coordinates and the following position coordinates; The queue arrangement subsystem calculates the distance between the position coordinates of each accompanying inspection mobile device and the position coordinates of the leading inspection mobile device based on the leading position coordinates and the following position coordinates, and establishes a logistics inspection queue and coding for multiple inspection mobile devices; Each patrol mobile device is equipped with a tracking processing subsystem, an image processing subsystem and an obstacle processing unit. The tracking processing subsystem uses a target locking algorithm and a distance keeping algorithm to perform real-time target locking and distance keeping along with the patrol mobile device; the image processing subsystem uses a visual sensor to obtain images of the surrounding environment in real time, uses a deep learning model to detect and locate foreign objects in each image, and identifies obstacles on the image; the obstacle processing unit maps the diameter of the obstacle with different preset obstacle diameter intervals, and selects the corresponding deflection angle to bypass the obstacle based on the mapping result.

8. The machine vision-based logistics inspection system according to claim 7, characterized in that: The tracking processing subsystem includes a target locking module and a distance keeping module; The target locking module calculates the coordinates of the center point of the rectangular area where the target to be followed is located in the field of view image and the center area of ​​the field of view image, and always keeps the coordinates of the center point of the rectangular area located in the center area of ​​the field of view image; The distance keeping module tracks the distance to the target through a distance keeping algorithm, calculates the difference between the real-time distance between the patrol mobile device and the target and the set optimal distance, sets the optimal difference interval, controls the difference value to remain within the optimal difference interval, and realizes distance keeping.

Images