A method, an internet of things device, and a system for collaborative processing of internet of things devices

Abstract

The application discloses a method for collaborative processing of Internet of Things devices, which comprises the following steps: obtaining a current image at any first Internet of Things device belonging to a same group, extracting image feature data, forming a first data packet by using the feature data, broadcasting the first data packet through a virtual network composed of all Internet of Things devices in the group, and enabling each Internet of Things device in the group to match the feature data in the first data packet with compared image data in an image sub-library stored by the Internet of Things device, wherein a union of image sub-libraries stored by each Internet of Things device in the group is a desired image library, and the desired image library comprises total data desired for compared images. The application creates a processing mode with lower cost and better scalability between single front-end device comparison and single back-end comparison.

Description

Technical Field

[0001] This invention relates to the field of distributed processing in the Internet of Things (IoT), and more particularly, to a method for collaborative processing of IoT devices. Background Technology

[0002] The Internet of Things (IoT) comprises a large number of IoT devices distributed in different physical locations. These IoT devices can be managed by an IoT management platform, which is typically referred to as the back-end device and the IoT devices as the front-end device.

[0003] IoT devices can be categorized by their functional applications, including attendance front-end devices, access control front-end devices, and so on. Taking image recognition-based attendance front-end devices as an example, such as various attendance devices distributed in a factory park, they mainly identify and authenticate the biometric images of a large number of people in the factory park, comparing the currently acquired biometric image with N frames of comparison images, i.e., a 1 to N comparison function.

[0004] Currently, based on the entity performing image comparison, there are two main types:

[0005] One method involves comparison on the front-end device. The comparison image is sent to the management platform (back-end device) and stored on the local front-end device. However, storing the comparison image on the local front-end device requires a lot of storage resources. Due to the limitations of the front-end device's capabilities, storage, and algorithms, the amount of comparison image data stored is limited. But this method has good interactive effects, is less affected by the network environment, and supports offline data processing.

[0006] Another approach involves comparison on a backend device. The frontend device sends the acquired images to the backend device for comparison. This method places high demands on the backend device's network, algorithms, and computing power. An anomaly in the backend device or network could paralyze the entire system. The advantage is that the images being compared are stored on the backend device, resulting in a larger total database capacity for the compared faces than any single frontend device's maximum data capacity. The amount of compared image data stored is also greater than that of the frontend device. However, since the backend device acts as the central unit, it is more expensive and has higher requirements for the network environment. Furthermore, the backend device needs to process all images from the frontend devices and return the results to each corresponding frontend device. This places high demands on the backend device's ability to handle multiple tasks simultaneously. If tasks experience queuing, even with a good network environment, the comparison speed at the backend will be slower than that of direct comparison by the frontend device.

[0007] Based on the advantages and disadvantages of the two comparison methods mentioned above, front-end equipment is usually used for comparison in general locations, such as residential areas, enterprises, schools, and government buildings, where the number of images being compared is less than 100,000 frames. Larger areas, such as large factories, generally use back-end equipment for comparison, where the number of images being compared is more than 100,000 frames, requiring the addition of back-end equipment or an increase in the computing power of the back-end equipment. Summary of the Invention

[0008] This invention provides a method for collaborative processing of Internet of Things (IoT) devices to achieve comparison and processing of massive image data at low cost.

[0009] The method for collaborative processing of IoT devices provided by this invention is implemented as follows:

[0010] On the side of any first IoT device belonging to the same group,

[0011] Acquire the current image, extract image feature data, and assemble the feature data into a first data packet.

[0012] The first data packet is broadcast through a virtual network consisting of all IoT devices in the group, enabling each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-database.

[0013] in,

[0014] The union of the image sub-libraries stored by each IoT device in the group is the desired image library, which includes the total data expected to be used for comparison of images.

[0015] Preferably, the first data packet also carries time information, which is used to identify the time when the current image was acquired.

[0016] The process of enabling each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-library includes:

[0017] Each IoT device in the group,

[0018] The received first data packet is stored in the queue of this IoT device.

[0019] Following the last-in-first-out (LIFO) principle, the current first data packet, which was the last one to enter the queue, is selected from the queue.

[0020] Based on the time information in the first data packet and the current time information, determine whether the first data packet exceeds the set first time threshold.

[0021] If so, discard the current data packet and process the next data packet in the queue.

[0022] Otherwise, the feature data in the current first data packet is matched with the image data to be compared in the image sub-library stored in this IoT device, and the matching result is returned;

[0023] The method further includes:

[0024] The first IoT device receives matching results from each IoT device in the group.

[0025] If more than one match is received, the match with the highest similarity will be taken as the final result.

[0026] If no matching results are received, the matching is deemed to have failed.

[0027] Preferably, the first data packet further includes a device identifier for identifying device information of the first IoT device, and a data packet identifier for distinguishing each first data packet.

[0028] The step of matching the feature data in the current first data packet with the image data to be compared in the image sub-library stored by this IoT device and returning the matching result includes:

[0029] If the similarity between the feature data in the current first data packet and the image data to be compared in the image sub-library stored by this IoT device is greater than a set first similarity threshold, then

[0030] The matching result and the packet identifier in the current first packet are combined to form the second packet.

[0031] Based on the device identifier in the first data packet, the second data packet is sent to the IoT device with the device identifier via a virtual network in a point-to-point communication manner.

[0032] Preferably, the first IoT device receives matching results returned by each IoT device in the group, including:

[0033] The first IoT device receives matching results from each IoT device in the group within a set second time threshold, and discards matching results that have exceeded the time limit.

[0034] After determining that the match has failed, the following further steps are included:

[0035] Reacquire the current image and extract its feature data. Compare this feature data with the previous feature data. If the similarity exceeds the set second similarity threshold, search for matching results received after the timeout.

[0036] The present invention also provides an Internet of Things (IoT) device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute steps of a method for collaborative processing of any of the IoT devices.

[0037] The present invention also provides an Internet of Things (IoT) system, including at least one or more IoT devices, the at least one or more IoT devices forming at least one group, each IoT device in the same group being connected through a virtual network, and the union of the image sub-libraries stored by each IoT device in the same group being a desired image library, the desired image library including the total data of images expected to be compared.

[0038] Any first IoT device in the same group is used to acquire the current image, extract image feature data, form the feature data into a first data packet, and broadcast the first data packet through a virtual network composed of all IoT devices in the group, so that each IoT device in the group matches the feature data in the first data packet with the image data to be compared in its own stored image sub-library.

[0039] Preferably, if the image sub-libraries stored by the IoT devices have the same capacity, then the number of IoT devices included in each unit group is: the rounded-up ratio of the expected total capacity of the image library to the maximum capacity of the image sub-library that a single IoT device can store;

[0040] If the image sub-libraries stored by the IoT devices have different capacities, then the number of IoT devices included in each unit group is: the rounded-up ratio of the expected total image library capacity to the minimum capacity of the image sub-libraries stored in all IoT devices.

[0041] The number of groups in the IoT system is the rounded-up ratio of the total number of IoT devices to the number of IoT devices included in a single group.

[0042] Preferably, each IoT device in the IoT system has a group identifier for identifying the group to which the IoT device belongs, and a device identifier for identifying the IoT device information. IoT devices with the same device identifier in each group have the same image sub-library.

[0043] If a group contains fewer IoT devices than other groups, the group shares the IoT devices corresponding to the missing device identifiers with any other group.

[0044] Preferably, the system further includes a platform device for managing IoT devices, which divides the data capacity of the desired image library into multiple image sub-libraries according to the capacity of the image sub-libraries stored by the IoT devices, and sends the image sub-libraries of the IoT devices to each IoT device according to the group identifier and the device identifier.

[0045] Preferably, the platform device configures other fault-free IoT devices with the same device identifier to share the fault in the event of a failure of any IoT device;

[0046] The virtual networks between different groups are isolated from each other, and each IoT device is connected to the platform device via a local area network; the idle time of each IoT device in the same group is different.

[0047] This application distributes image data comparison tasks to various IoT devices in the same group. The comparison is performed independently through the image sub-libraries stored by each IoT device in the group. In this way, each image comparison is performed by multiple IoT devices and completes the comparison of all images to be compared in the desired image library. Compared with a single IoT device, it improves computing power and expands the image library capacity of the images to be compared. It creates a processing mode with lower implementation cost, better scalability and higher flexibility between comparison by a single front-end device and comparison by a single back-end device. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating a process for implementing collaborative processing of IoT devices in this application.

[0049] Figure 2 A diagram illustrating the grouping of various front-end devices.

[0050] Figure 3 A flowchart illustrating a collaborative process for any front-end device within a group.

[0051] Figure 4 This is a schematic diagram of an Internet of Things (IoT) device according to an embodiment of this application.

[0052] Figure 5 This is another schematic diagram of an IoT device according to an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0054] This application divides a desired image library containing total data intended for comparison into several image library sub-libraries, which are stored in various Internet of Things (IoT) devices. The IoT devices storing the image library sub-libraries are grouped together, and the comparison processing of image data is achieved through the collaborative processing of the various IoT devices within the group.

[0055] See Figure 1 As shown, Figure 1This is a schematic diagram illustrating a process for implementing collaborative processing of IoT devices according to this application. On the side of any first IoT device belonging to the same group, including:

[0056] Step 101: Acquire the current image, extract image feature data, and assemble the feature data into a first data packet.

[0057] Step 102: The first data packet is broadcast through a virtual network composed of all IoT devices in the group, so that each IoT device in the group matches the feature data in the first data packet with the image data to be compared in its own stored image sub-database.

[0058] in,

[0059] The union of the image sub-libraries stored by each IoT device in the group is the desired image library, which includes the total data expected to be used for comparison of images.

[0060] As can be seen from the above processing, for a single data comparison, the group method realizes the data comparison of all the images to be compared in the desired image library, which is equivalent to expanding the database. At the same time, the collaborative processing of multiple IoT devices is equivalent to improving the computing power of the comparison. Between comparison by a single front-end device and comparison by a single back-end device, a processing mode with lower implementation cost and better scalability is created, which solves the pain point of excessively high cost caused by directly introducing back-end devices at a low cost.

[0061] The following example uses a facial recognition-based access control device to illustrate the method of collaborative work among front-end devices. It should be understood that front-end devices are not limited to access control devices, but include various other IoT devices, such as attendance systems and monitoring devices. The data to be processed is not limited to facial image data, but can also include other data, such as iris image data and identity information data.

[0062] For a face recognition-based access control system, there are Duse front-end devices, where Duse is a natural number greater than 1. Several front-end devices are grouped together such that the sum of the capacities of the face image databases stored by each front-end device in the group is equal to the total capacity U of the face image database to be compared, i.e., the total capacity of the desired image database, thus obtaining more than one group.

[0063] Assuming all front-end devices have the same capacity for storing face images, and the maximum capacity of the face image library on each front-end device is Amax, then the number of devices D in a unit group is: the ratio of the expected total capacity of the image library to the maximum capacity of the front-end devices, rounded up, expressed mathematically as:

[0064]

[0065] Among them, symbols This indicates rounding up to the nearest integer.

[0066] Given that the capacity of the face image database that each front-end device can store varies, to improve the collaborative work of the group, preferably, based on the capacity of the face image database that each front-end device can store, the smallest capacity value is used as the capacity A of a single front-end device for the group. Then, the number D of front-end devices in a unit group is: the ratio of the expected total capacity of the image database to the smallest capacity value among the front-end devices, rounded up, expressed mathematically as:

[0067]

[0068] Therefore, the total number of groups M is the rounded-up ratio of the total number of front-end devices Duse in the access control system to the number of devices in a single group, expressed mathematically as:

[0069]

[0070] Based on the number of groups and the number of front-end devices in each group, all front-end devices in the access control system are grouped. Ideally, front-end devices that are idle at different times are configured in the same group, such as front-end devices for controlling entrances and front-end devices for controlling exits. This can better utilize idle computing power and avoid all front-end devices in the same group being used to control the entrance direction, thereby avoiding a large number of tasks being submitted during working hours and all computing power being idle during off-hours.

[0071] In this way, the various front-end devices in the access control system are divided into front-end devices belonging to different groups. To facilitate the differentiation of each front-end device, each front-end device is identified by its group number and the device number within that group, resulting in a two-dimensional array of group identifiers and device identifiers, denoted as N[m, d], where m is the group identifier, d is the device identifier, and d is a natural number that is 0 ≤ m ≤ M-1 and 0 ≤ d ≤ D-1. See also... Figure 2 As shown, Figure 2 This is a diagram illustrating the grouping of various front-end devices. Each group contains D front-end devices, and there are a total of M groups. Each front-end device has its own group number and device number.

[0072] The platform device used to manage various front-end devices acts as the back-end device. For each front-end device within the same group, the total desired image library capacity U is divided according to the capacity of each individual front-end device, resulting in a face image sub-library. Based on a two-dimensional array identifier, these sub-libraries are distributed to each front-end device. Thus, the union of the face image sub-libraries across all front-end devices in the same group equals the total desired image library capacity U.

[0073] U=N[0,0]∪N[0,1]∪….∪N[0,d1]…..∪N[0,D-1];

[0074] U=N[1,0]∪N[1,1]∪….∪N[1,d1]…..∪N[1,D-1];

[0075] U=N[2,0]∪N[2,1]∪….∪N[2,d1]…..∪N[2,D-1];

[0076] ...

[0077] U=N[M-1,0]∪N[M-1,1]∪….∪N[M-1,d1]…..∪N[M-1,D-1];

[0078] Since M is rounded up to obtain the total number of device groups, the total number of devices in the last group may be less than D. Therefore, the union of the face image sub-libraries of all front-end devices in the last group M-1 may not constitute the complete expected total image library capacity U. Therefore, the device numbers in the previous group M-2 are compared with the device numbers in group M-1. The missing device numbers in M-1 are mapped to the device numbers in the previous group M-1, and the face image sub-libraries of the corresponding front-end devices are shared with group M-1. This ensures that in each group, the union of the face image sub-libraries formed by each front-end device in that group can constitute the complete expected total image library capacity U.

[0079] Assuming the total number of front-end devices in the last group M-1 is x, then the number of front-end devices shared in the previous group M-2 is the difference between the number of front-end devices D in the unit group and the total number of front-end devices in group M-1, expressed mathematically as Dx, with the corresponding device numbers being Dx-1 to D-1.

[0080] Therefore, for the last group M-1, the union of the face image sub-databases of all front-end devices in this group is:

[0081] U=N[M-1,0]∪N[M-1,1]∪….∪N[M-1,x-1]∪N[M-2,x]∪N[M-2,Dx-1]∪N[M-2,Dx]…..∪N[M-2,D-1]

[0082] For front-end devices with the same device ID in different groups, the facial image sub-databases stored in these front-end devices are identical. In other words, if the front-end devices have the same device ID, then the facial image sub-databases stored on these front-end devices are identical. This can be expressed mathematically as:

[0083] N[0,0]=N[1,0]=N[2,0]=……=N[M-1,0];

[0084] N[0,1]=N[1,1]=N[2,1]=……=N[M-1,1];

[0085] ...

[0086] N[0, D-1]=N[1, D-1]=N[2, D-1]=……=N[M-1, D-1];

[0087] like Figure 2 As shown in the figure, the face image library is illustrated from the dimensions of group and device number. Within the same group, the union of the face image sub-libraries of each front-end device is U. In each group, the face image sub-libraries of front-end devices with the same device number are the same, while the face image sub-libraries of front-end devices with different device numbers are different.

[0088] All front-end devices in the system are connected to the established local area network (LAN), and each front-end device is connected to the platform device through the LAN. Based on the established LAN, the platform device configures virtual addresses for each front-end device in each group, thereby establishing a virtual LAN for that group. In this way, messages can be broadcast or transmitted point-to-point within the same group based on the virtual address. Different groups have different virtual LANs, and the virtual LANs of each group are isolated from each other.

[0089] For situations where group M-1 shares a front-end device with a device ID not included in group M-2, the various front-end devices in group M-1, along with the shared front-end devices in group M-2, can form a virtual local area network (VLAN) for group M-1. When a front-end device in a group malfunctions, sharing with a front-end device in another group with the same device ID can prevent incomplete face image databases from causing abnormal comparison results. Specifically, when a front-end device fails, the platform device can configure a working front-end device with the same device ID as the failed device to be shared. In other words, the shared front-end device is configured to belong to both its existing VLAN and the VLAN where the failed front-end device resides, thus belonging to two VLANs simultaneously.

[0090] Alternatively, different virtual LANs can be connected through a front-end device that functions as a gateway, allowing different groups of virtual networks to be isolated or to transmit specific messages through a specific front-end device.

[0091] By establishing groups and virtual local area networks for each group, from the group's perspective, the collaborative work of various front-end devices within the group is equivalent to expanding the facial image library of any front-end device within the group.

[0092] The tasks of front-end devices in an access control system typically include: acquiring the current image, extracting feature data based on the current image, comparing the feature data, and controlling the system based on the comparison results. Since feature data comparison is the most time-consuming and resource-intensive task, it is distributed to all front-end devices in the group for processing. Furthermore, given that the quality of the acquired face image may be poor, or multiple images may be acquired simultaneously, leading to failures in facial feature extraction, feature extraction is performed on the device acquiring the current face image, while the feature data comparison task is distributed to each front-end device in the group. This reduces the transmission of source image data and facilitates human-computer interaction at the front-end devices.

[0093] See Figure 3 As shown, Figure 3 This is a flowchart illustrating collaborative processing for any front-end device within a group. For ease of description, the front-end device acquiring the current face image will be referred to as the first front-end device. This collaborative processing includes:

[0094] Step 301: The first front-end device acquires the current face image, extracts face image feature data, and at least assembles the feature data into a first data packet, which is then broadcast in the virtual local area network where the first front-end device is located, that is, broadcast within the group to which the first front-end device belongs.

[0095] Preferably, the first data packet may further include time information for acquiring the current image, and / or, a device identifier for acquiring the current image stored in a queue, wherein the time information is timestamp information determined by the system, and the device identifier is device number information. Preferably, the first data packet further includes an identifier for distinguishing each first data packet, so that other front-end devices in the group will return the processing result of the data packet accordingly.

[0096] Preferably, the first front-end device stores the first data packet in its own queue so that it can perform feature data comparison.

[0097] Step 302: After receiving the first data packet broadcast, each of the other front-end devices in the group stores the first data packet in its own queue and processes the first data packet in the queue according to the principle of last-in, first-out.

[0098] Each front-end device within the group (including the first front-end device) processes the first packet in the queue, including:

[0099] Determine if there is a first data packet being processed.

[0100] If so, process the first data packet. After the first data packet is processed, select the last data packet that entered the queue from the queue.

[0101] Otherwise, following the Last-In-First-Out (LIFO) principle, the first data packet at the tail of the queue is selected for processing. The queue stores first data packets in chronological order of reception time, with earlier received packets at the head of the queue and later received packets at the tail.

[0102] Based on the time information in the first data packet and the current time, determine whether the first data packet exceeds a set first time threshold.

[0103] If no timeout occurs, the feature data in the first data packet will be compared with all the face images to be compared in the face image sub-database stored in this device.

[0104] If the similarity is greater than the set first similarity threshold, it means that the face image to be compared is matched in the face image sub-library stored in this device. The matching result is extracted, and the matching result is returned to the front-end device with the device information in a point-to-point transmission manner according to the device identifier in the first data packet. Preferably, the matching result and the data packet identifier in the first data packet are combined into a second data packet, and the second data packet is returned to the corresponding front-end device.

[0105] Preferably, if the similarity is not greater than a set first similarity threshold, it indicates that no matching face image has been found in the face image sub-database stored in this device, and a matching failure notification is returned to the front-end device with the device information via point-to-point transmission.

[0106] If a timeout occurs, the packet is discarded, and the next packet at the end of the queue is processed.

[0107] Step 303: Within a set second time threshold, the first front-end device receives matching results from other front-end devices in the group.

[0108] If multiple matching results are received within the second time threshold, the matching result with the highest similarity is selected for output display, event logging, access control, etc.

[0109] If only one matching result is received within the second time threshold, output display, event logging, access control, etc. are performed based on that matching result;

[0110] If no matching result is received within the second time threshold, the comparison is deemed to have failed.

[0111] Preferably, given that the time spent waiting for matching results is a major factor affecting user experience, reducing this time can improve the user experience. Furthermore, some timed-out matching results from front-end devices in the group are discarded, resulting in matching results that should have appeared as failed matches. Therefore, when a match is determined to have failed:

[0112] Step 304: The first front-end device outputs a prompt to reacquire the current image. After the current image is reacquired and feature data is extracted, the similarity between the previously acquired images is compared. If the similarity exceeds the set second similarity threshold, a search is performed from the matching results received after the timeout. If a matching result is found, output display, event logging, access control, etc. are performed based on the matching result. In this way, it is not necessary to broadcast data packets to other devices in the group again, so as to reduce the time spent waiting for the matching result.

[0113] In the above process:

[0114] The time taken to acquire the current image and extract its feature data is t1.

[0115] The time taken for the first front-end device to send a broadcast data packet to any front-end device in the group and receive the broadcast data packet is t2.

[0116] The time taken for each front-end device in the group to perform the comparison is t3.

[0117] The time taken for each front-end device in the group to return the matching result to the first front-end device is t4.

[0118] The total process time is t = t1 + t2 + t3 + t4;

[0119] The sum of t2 and t4 represents the network latency, while the sum of t1 and t3 represents the feature extraction and comparison time, which can be considered the algorithm's execution time, typically around 300ms. Based on network latency, 3G network latency is above 300ms×2, 4G network latency is above 70ms×2, and 5G network latency is above 10ms×2. Therefore, in a high-speed network environment, a single network latency is equivalent to one-fifteenth of the algorithm's execution time. Even with network congestion, under high-speed network conditions, even without considering the reduction in time consumption of the distributed comparison algorithm in this application, compared to the current experience where a complete face comparison takes more than 1.5 seconds, users will perceive a slower comparison speed. The additional network latency compared to the comparison time of a single front-end access control face recognition device is acceptable.

[0120] Step 305: Other front-end devices in the group wait to receive the next broadcast first data packet.

[0121] This embodiment utilizes the divided groups to establish a virtual desired image library, achieving several times the capacity expansion compared to a single front-end device. Simultaneously, by dividing the desired image library into several image sub-libraries, the download speed is faster than with a single front-end device, achieving low-cost expansion of the image library and solving the problem of low image library capacity on front-end devices. It also fully utilizes idle front-end devices at different times, improving the computing power for each data comparison. Furthermore, with the development of 5G networks and increasingly lower network latency, this application has even better application prospects.

[0122] See Figure 4 As shown, Figure 4 This is a schematic diagram of an IoT device according to an embodiment of this application. The IoT device includes,

[0123] The image acquisition module is used to acquire the current image.

[0124] The feature extraction module is used to extract image feature data.

[0125] The broadcast module is used to assemble the feature data into a first data packet and broadcast the first data packet through a virtual network composed of all IoT devices in the first group to which this IoT device belongs.

[0126] The comparison module is used to match feature data from the first data packet from the feature extraction module of this IoT device, or from the first data packet from other IoT devices in the first group, with the image data to be compared in the image sub-library stored by this IoT device.

[0127] In this group, the union of the image sub-libraries stored by each IoT device in the first group is the desired image library.

[0128] Preferably, the IoT device also includes,

[0129] The transceiver module sends the matching results of this IoT device, receives matching results returned from other IoT devices, and the first data packet broadcast by other IoT devices.

[0130] See Figure 5 As shown, Figure 5 This is another schematic diagram of an IoT device according to an embodiment of this application. The IoT device includes a memory and a processor, the memory storing a computer program, and the processor being configured to execute steps of a method for collaborative processing of any of the IoT devices.

[0131] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0132] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0133] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned steps of collaborative processing of IoT devices.

[0134] For the device / network-side equipment / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0135] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for collaborative processing of Internet of Things (IoT) devices, characterized in that, The method includes: On the side of any first IoT device belonging to the same group, The current image is acquired, image feature data is extracted, and the feature data is assembled into a first data packet. The first data packet carries time information, which is used to identify the time when the current image was acquired. The first data packet is broadcast through a virtual network consisting of all IoT devices in the group, enabling each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-database. Receive matching results from various IoT devices in the group. If more than one match is received, the match with the highest similarity will be taken as the final result. in, The union of the image sub-libraries stored by each IoT device in the group is the desired image library, which includes the total data expected to be used for comparison of images. The process of enabling each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-library includes: Each IoT device in the group stores the first data packet it receives in its own IoT device queue. Select the last data packet to enter the queue, which is currently the first data packet. Based on the time information in the first data packet and the current time information, determine whether the first data packet exceeds the set first time threshold. If so, discard the current first data packet and process the next first data packet in the queue; otherwise, match the feature data in the first data packet with the image data to be compared in the image sub-library stored by this IoT device.

2. The method as described in claim 1, characterized in that, The method further includes: If no matching results are received, the matching is deemed to have failed.

3. The method as described in claim 1 or 2, characterized in that, The first data packet also includes a device identifier for identifying device information of the first IoT device, and a data packet identifier for distinguishing each first data packet. The step of matching the feature data in the first data packet with the image data to be compared in the image sub-library stored by the Internet of Things device includes: If the similarity between the feature data in the current first data packet and the image data to be compared in the image sub-library stored by this IoT device is greater than a set first similarity threshold, then The matching result and the packet identifier in the current first packet are combined to form the second packet. Based on the device identifier in the first data packet, the second data packet is sent to the IoT device with the device identifier via a virtual network in a point-to-point communication manner.

4. The method as described in claim 2, characterized in that, The first IoT device receives matching results returned by each IoT device in the group, including: The first IoT device receives matching results from each IoT device in the group within a set second time threshold, and discards matching results that have exceeded the time limit. After determining that the match has failed, the following further steps are included: Reacquire the current image and extract its feature data. Compare this feature data with the previous feature data. If the similarity exceeds the set second similarity threshold, search for matching results received after the timeout.

5. An Internet of Things (IoT) device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor being configured to, when executed, implement the steps of the method for collaborative processing of IoT devices as described in any one of claims 1 to 4.

6. An Internet of Things (IoT) system, comprising at least one IoT device, characterized in that, The at least one Internet of Things (IoT) device forms at least one group, and the IoT devices in the same group are connected through a virtual network. The union of the image sub-libraries stored by the IoT devices in the same group is a desired image library, which includes the total data of images expected to be compared. Any first IoT device in the same group is used to acquire the current image, extract image feature data, assemble the feature data into a first data packet, and broadcast the first data packet through a virtual network composed of all IoT devices in the group. This allows each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-library, and receive the matching results returned by each IoT device in the group. If more than one matching result is received, the matching result with the highest similarity is taken as the final result. in, The first data packet carries time information, which is used to identify the time when the current image was acquired. The process of enabling each IoT device in the group to match the feature data in the first data packet with the image data to be compared in its own stored image sub-library includes: Each IoT device in the group stores the first data packet it receives in its own IoT device queue. Select the last data packet to enter the queue, which is currently the first data packet. Based on the time information in the first data packet and the current time information, determine whether the first data packet exceeds the set first time threshold. If so, discard the current first data packet and process the next first data packet in the queue; otherwise, match the feature data in the first data packet with the image data to be compared in the image sub-library stored by this IoT device.

7. The system as described in claim 6, characterized in that, If the image sub-libraries stored by the IoT devices have the same capacity, then the number of IoT devices included in each unit group is: the rounded-up ratio of the expected total capacity of the image library to the maximum capacity of the image sub-library that a single IoT device can store. If the image sub-libraries stored by the IoT devices have different capacities, then the number of IoT devices included in each unit group is: the rounded-up ratio of the expected total image library capacity to the minimum capacity of the image sub-libraries stored in all IoT devices. The number of groups in the IoT system is the rounded-up ratio of the total number of IoT devices to the number of IoT devices included in a single group.

8. The system as described in claim 6 or 7, characterized in that, Each IoT device in the IoT system has a group identifier for identifying the group to which it belongs, and a device identifier for identifying the IoT device's information. IoT devices with the same device identifier in each group have the same image sub-library. If a group contains fewer IoT devices than other groups, the group shares the IoT devices corresponding to the missing device identifiers with any other group.

9. The system as described in claim 8, characterized in that, The system also includes a platform device for managing IoT devices. The platform device divides the data capacity of the desired image library into multiple image sub-libraries according to the capacity of the image sub-libraries stored by the IoT devices, and sends the image sub-libraries of the IoT devices to each IoT device according to the group identifier and device identifier.

10. The system as described in claim 9, characterized in that, In the event of a failure of any IoT device, the platform device will configure other fault-free IoT devices with the same device identifier to share the configuration. The virtual networks between different groups are isolated from each other, and each IoT device is connected to the platform device via a local area network; the idle time of each IoT device in the same group is different.

Images