An edge cloud cooperative image change detection method and system

Abstract

The application discloses an edge cloud cooperative image change detection method and system. The edge cloud cooperative image change detection method comprises the following steps: step 1, acquiring a to-be-detected image; step 2, dividing the to-be-detected image into a plurality of single images; step 3, acquiring image features of the single images; step 4, assigning a serial number to each image feature and sending each image feature to an edge server; step 5, acquiring a graph element comparator; step 6, inputting each image feature into the graph element comparator to acquire a label output by the graph element comparator, wherein the label comprises a change label; step 7, assigning a new serial number to each image feature that has not been input into the graph element comparator after the graph element comparator generates the change label; and step 8, repeating steps 6 to 7 until the end. The edge cloud cooperative image change detection method has a faster response time and can save computing power compared with the prior art.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an edge-cloud collaborative image change detection method and an edge-cloud collaborative image change detection system. Background Technology

[0002] In existing image change detection, a fixed step size is usually used as the window to detect the image. In actual detection, the image to be detected is divided into multiple parts of the same size as the window, and then detected part by part in turn.

[0003] However, in most cases, image change detection methods are used in monitoring fields, such as monitoring whether someone is approaching on the ground from a communication tower. Typically, such monitoring is unlikely to involve a large area in an image; the change may only appear in a local area. It is unnecessary to detect the entire image, thus wasting computing power. In particular, such image detection is usually performed on edge servers (e.g., servers installed on the tower). This means that performing a global comparison for every frame would be a waste of computing power and time.

[0004] Therefore, there is a need for a technical solution to address or at least mitigate the aforementioned shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to provide an image change detection method with edge-cloud collaboration to overcome or at least mitigate one of the above-mentioned defects of the prior art.

[0006] One aspect of the present invention provides an edge-cloud collaborative image change detection method, the edge-cloud collaborative image change detection method comprising:

[0007] Step 1: The edge server acquires the image to be detected and sends it to the central cloud server;

[0008] Step 2: The central cloud server segments the image to be detected into multiple individual images;

[0009] Step 3: The central cloud server acquires the image features of each individual image;

[0010] Step 4: The central cloud server sorts the image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server;

[0011] Step 5: The edge server obtains the trained primitive comparator;

[0012] Step 6: The edge server inputs each image feature into the primitive comparator in ascending order of sequence number, thereby obtaining the labels output by the primitive comparator, including change labels;

[0013] Step 7: After the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to the image features that have not yet been input into the primitive comparator.

[0014] Step 8: Repeat steps 6 to 7 until the number of obtained change labels exceeds a preset value or all image features are input into the primitive comparator.

[0015] Optionally, the central cloud server sorts the image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server, including:

[0016] The central cloud server obtains the trained object classifier;

[0017] The central cloud server inputs the image features of each individual image in sequence into the object classifier to obtain object classification labels;

[0018] Obtain a first sequence database, which includes at least one preset object category label and a sequence number group. A sequence number group and a preset object category label, a sequence number group includes multiple different sequence numbers, and the sequence numbers in each sequence number group are different.

[0019] The following operations are performed on each individual image in the order they enter the object classifier:

[0020] Obtain object classification labels based on image features of individual images;

[0021] Extract a sequence number from the sequence number group corresponding to the preset object classification label that is the same as the object classification label, and use it as the sequence number of the image feature of the single image.

[0022] Optionally, retrieving one of the sequence numbers corresponding to the preset object classification label that is the same as the object classification label as the sequence number of the image feature of the single image includes:

[0023] Take the largest sequence number from the sequence number group and use it as the sequence number of the image feature of the single image.

[0024] Optionally, the object classification labels include water surface labels and road labels;

[0025] The smallest sequence number in the sequence number group corresponding to the road label is greater than the largest sequence number in the sequence number group corresponding to the water surface label.

[0026] Optionally, the image change detection method further includes:

[0027] The object classification labels further include building labels;

[0028] The smallest sequence number in the sequence number group corresponding to the water surface label is greater than the largest sequence number in the sequence number group corresponding to the building label.

[0029] Optionally, the image change detection method further includes:

[0030] The object classification label further includes a tree branch label;

[0031] The smallest sequence number in the sequence number group corresponding to the building label is greater than the largest sequence number in the sequence number group corresponding to the tree branch label.

[0032] Optionally, after the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to each image feature that has not yet been input into the primitive comparator, including:

[0033] Obtain the individual image corresponding to the image features of the change label;

[0034] Obtain the position of the individual image within the image to be detected;

[0035] Obtain object classification labels from individual images;

[0036] Obtain a preset linkage database, which includes at least one preset object classification label and linkage range, with one linkage range corresponding to one preset object classification label.

[0037] Obtain the associated range corresponding to the preset object category label that is the same as the object category label;

[0038] Based on the linkage range, the image features that have not yet been input into the primitive comparator are reordered, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator.

[0039] Optionally, the linkage range includes a first layer of linkage information, which includes at least one pixel position information and a sequence number, with one sequence number corresponding to one pixel position information.

[0040] The step of reordering the image features that have not yet been input into the primitive comparator according to the linkage range, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator, includes:

[0041] Determine whether the individual image containing the pixel position information in the first layer of linkage information has already been input into the primitive comparator. If not, then...

[0042] Retrieve single-unit images that have not been input into a primitive comparator;

[0043] For each individual image that has not been input into a primitive comparator, perform the following operations:

[0044] The index corresponding to the pixel position information of a single image that has not been input into the primitive comparator is obtained as the new index of the image feature of that single image.

[0045] Optionally, the linkage range includes second-layer linkage information, which includes multiple sequence numbers;

[0046] Obtain image features of other single images besides the individual images that have been input into the primitive comparator and the single images corresponding to the image features with the new sequence number;

[0047] The second layer of linked information is used to randomly assign new serial numbers to the image features of each individual image other than those already input to the primitive comparator and the individual images corresponding to the image features that have obtained new serial numbers.

[0048] This application also provides an edge-cloud collaborative image change detection system, which includes a central cloud server and an edge server. The central cloud server and the edge server cooperate to implement the edge-cloud collaborative image change detection method as described above.

[0049] Beneficial effects

[0050] The edge-cloud collaborative image change detection method of this application divides image change detection into two parts. The first part runs in the central cloud server to find hot spots (parts in the image that may change) and sort them according to the hot spots. The second part runs on the edge server to identify the image features sorted by the central cloud server. This increases the probability of detecting changes in the first few identifications, allowing the method to stop or perform other subsequent steps (such as issuing an alarm when a change is detected) after a change is detected. This results in a faster response and saves computing power. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the image change detection method for edge-cloud collaboration according to the first embodiment of this application.

[0052] Figure 2 This is a schematic diagram of the image to be detected in the edge-cloud collaborative image change detection method of the first embodiment of this application;

[0053] Figure 3 Example of a scenario using the edge-cloud collaborative image change detection method of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0055] Figure 1 This is a flowchart illustrating the edge-cloud collaborative image change detection method according to the first embodiment of this application.

[0056] like Figure 1 The edge-cloud collaborative image change detection method shown includes:

[0057] Step 1: The edge server acquires the image to be detected and sends it to the central cloud server;

[0058] Step 2: The central cloud server segments the image to be detected into multiple individual images;

[0059] Step 3: The central cloud server acquires the image features of each individual image;

[0060] Step 4: The central cloud server sorts the image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server;

[0061] Step 5: The edge server obtains the trained primitive comparator;

[0062] Step 6: The edge server inputs each image feature into the primitive comparator in ascending order of sequence number, thereby obtaining the labels output by the primitive comparator, including change labels;

[0063] Step 7: After the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to the image features that have not yet been input into the primitive comparator.

[0064] Step 8: Repeat steps 6 to 7 until the number of obtained change labels exceeds a preset value or all image features are input into the primitive comparator.

[0065] The edge-cloud collaborative image change detection method of this application divides image change detection into two parts. The first part runs in the central cloud server to find hot spots (parts in the image that may change) and sort them according to the hot spots. The second part runs on the edge server to identify the image features sorted by the central cloud server. This increases the probability of detecting changes in the first few identifications, allowing the method to stop or perform other subsequent steps (such as issuing an alarm when a change is detected) after a change is detected. This results in a faster response and saves computing power.

[0066] Step 1: The edge server acquires the image to be detected and sends it to the central cloud server. For example, most images captured by cameras are first transmitted to the edge server. Taking tower monitoring as an example, the edge server is usually installed on the tower and connected to the camera. At this time, each frame of the image acquired by the camera is transmitted to the edge server as the image to be detected, and the edge server then transmits each frame of the image to be detected to the central server.

[0067] Step 2: The central cloud server segments the image to be detected into multiple individual images. For example, each video frame can be segmented with a basic segmentation unit of 16×16 pixels. It is understood that the size of the basic segmentation unit can be set as needed.

[0068] Step 3: The central cloud server obtains the image features of each individual image. The image features of each individual image can be extracted through feature extraction.

[0069] Step 4: The central cloud server sorts the image features according to the first-order generation logic, assigns a sequence number to each image feature, and sends each image feature to the edge server. For example, suppose an image is divided into 8 parts (most images are divided into more than 8 individual images; for example, a 1024*1024 image, if divided into 16×16 pixels, will have 64 individual images, but for the sake of simplicity, examples with too many individual images will not be described), then there are also 8 image features. Therefore, according to the first-order generation logic, each image feature is assigned a sequence number.

[0070] Specifically, for example, the individual images include a first individual image, a second individual image, a third individual image, a fourth individual image, a fifth individual image, a sixth individual image, a seventh individual image, and an eighth individual image. It is understood that there is a relative positional relationship between these individual images. For example, see [link to relevant documentation]. Figure 3 ,exist Figure 3 In the illustrated embodiment, each cell represents a single image. Figure 3 The illustrated embodiment contains 16 individual images. Each individual image is segmented from left to right and from top to bottom. The first individual image is... Figure 3 The square shown in Figure 1.

[0071] Each image feature is assigned a sequence number according to the first order generation logic. For example, the sequence number of the first single image is 1, the sequence number of the second single image is 4, the sequence number of the third single image is 3, the sequence number of the fourth single image is 2, the sequence number of the fifth single image is 7, the sequence number of the sixth single image is 6, the sequence number of the seventh single image is 6, and the sequence number of the eighth single image is 8. The following description will describe in detail how the first order generation logic is used for sorting, so it will not be repeated here.

[0072] In this embodiment, the central cloud server sorts the various image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server, including:

[0073] The central cloud server obtains the trained object classifier;

[0074] The central cloud server inputs the image features of each individual image in sequence into the object classifier to obtain object classification labels;

[0075] Obtain a first sequence database, which includes at least one preset object category label and a sequence number group. A sequence number group and a preset object category label, a sequence number group includes multiple different sequence numbers, and the sequence numbers in each sequence number group are different.

[0076] The following operations are performed on each individual image in the order they enter the object classifier:

[0077] Obtain object classification labels based on image features of individual images;

[0078] Extract a sequence number from the sequence number group corresponding to the preset object classification label that is the same as the object classification label, and use it as the sequence number of the image feature of the single image.

[0079] Taking the above example, the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh, twelfth, thirteenth, fourteenth, fifteenth, and sixteenth individual images obtained above are sequentially input into the trained object classifier to obtain the object classification label for each individual image. See [link to relevant documentation]. Figure 3 ,For example,

[0080] Figure 3 The first single-unit image shown ( Figure 3 The object category label for 1) is tree branch;

[0081] Second single-unit image ( Figure 3 The object category label for 2) is tree branch;

[0082] Third single-unit image ( Figure 3 The object category label for 3) is "wall".

[0083] Fourth single image ( Figure 3 The object category label for 4) is "wall".

[0084] Fifth single image ( Figure 3 The object category label for 5) is tree branch;

[0085] The sixth single image ( Figure 3 The object category label for 6) is tree branch;

[0086] Seventh single image ( Figure 3 The object category label for 7) is "wall".

[0087] Eighth single image ( Figure 3 The object category label for item 8) is "wall".

[0088] Ninth Monomer Image ( Figure 3 The object category label for 9) is chair;

[0089] The tenth single image ( Figure 3 The object category label for 10) is chair;

[0090] Eleventh single image ( Figure 3 The object category label for item 11) is "bed";

[0091] Twelfth Monograph Image ( Figure 3 The object category label for item 12) is "bed";

[0092] The thirteenth single image ( Figure 3 The object category label for 13) is floor;

[0093] Fourteenth individual image ( Figure 3 The object category label for 14) is floor;

[0094] The fifteenth individual image ( Figure 3 The object category label for item 15 is "bed";

[0095] Sixteenth individual image ( Figure 3 The object category label for 16 is "bed".

[0096] It is understandable that the object classifier is trained in advance, for example, Figure 3 The room shown is known to all its furnishings during monitoring. By training with a large dataset and manually labeling the images to form a training set, the category of each object can be obtained.

[0097] Once the category is obtained, a first sequence database is obtained. The first sequence database includes at least one preset object category label and a sequence number group. A sequence number group and a preset object category label are used. A sequence number group includes multiple different sequence numbers, and the sequence numbers in each sequence number group are different.

[0098] For example, the preset object classification labels can include various labels such as water surface labels, road labels, building labels, tree branch labels, bed labels, and seat labels. Each label is provided with a sequence number group. The sequence numbers in each sequence number group are all different from each other, and the sequence numbers in each sequence number group will not be the same as the sequence numbers in other sequence number groups. That is, all sequence numbers in all sequence number groups have one and only one, and there can be no repetition.

[0099] by Figure 3For example, suppose the sequence number groups for tree branch labels include (99, 98, 97, 96, 95, 94, 93, 92, 91, etc.), the sequence number groups for wall labels include (89, 88, 87, 86, 85, 84, 83, 82, 81, etc.), the sequence number groups for floor labels include (79, 78, 77, 76, 75, 74, 73, 72, 71, etc.), the sequence number groups for chair labels include (69, 68, 67, 66, 65, 64, 63, 62, 61, etc.), and the sequence number groups for bed labels include (59, 58, 57, 56, 55, 54, 53, 52, 51, etc.).

[0100] The following operations are performed on each individual image in the order they enter the object classifier:

[0101] Obtain object classification labels based on image features of individual images;

[0102] Extract a sequence number from the sequence number group corresponding to the preset object classification label that is the same as the object classification label, and use it as the sequence number of the image feature of the single image.

[0103] by Figure 3 For example, if the order of images entering the object classifier is: first single image, second single image, third single image, fourth single image, fifth single image, sixth single image, seventh single image, eighth single image, ninth single image, tenth single image, eleventh single image, twelfth single image, thirteenth single image, fourteenth single image, fifteenth single image, and sixteenth single image, then one of the sequence numbers corresponding to the preset object classification label that is the same as the object classification label is taken as the sequence number of the image feature of that single image. For example, the sequence number of the image feature of the first single image is 99, the sequence number of the image feature of the second single image is 92, and the sequence number of the image feature of the third single image is 89. The image feature numbers for the fourth single image are 89, the fifth single image are 96, the sixth single image are 93, the seventh single image are 88, the eighth single image are 87, the ninth single image are 68, the tenth single image are 69, the eleventh single image are 59, the twelfth single image are 58, the thirteenth single image are 78, the fourteenth single image are 77, the fifteenth single image are 55, and the sixteenth single image are 54.

[0104] It is understandable that within the same sequence number group, numbers can be randomly selected, or the largest sequence number in the group can be used as the sequence number of the image feature for that single image. For example, taking the image feature numbers of the first single image and the second single image as examples, if the largest sequence number in the group is used as the image feature number for that single image, then the image feature number of the first single image would be 99 and the image feature number of the second single image would be 98.

[0105] After retrieving these serial numbers, step 6: The edge server inputs each image feature into the primitive comparator in ascending order of serial number, thereby obtaining the labels output by the primitive comparator, including change labels. Figure 3 For example, the smallest sequence number is 54 for the image feature of the sixteenth individual image. Therefore, first input the image feature of the sixteenth individual image, and then input the image feature of the fifteenth individual image.

[0106] Step 7: After the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to the image features that have not yet been input into the primitive comparator.

[0107] For example, if the primitive comparator generates a change label after inputting the image features of the fifteenth unit image, indicating that the fifteenth unit image has changed, then the following steps are performed:

[0108] Step 7: After the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to the image features that have not yet been input into the primitive comparator.

[0109] Specifically, after the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to each image feature that has not yet been input into the primitive comparator, including:

[0110] Obtain the individual images corresponding to the image features of the change labels;

[0111] Obtain the position of the individual image within the image to be detected;

[0112] Obtain object classification labels from individual images;

[0113] Obtain a preset linkage database, which includes at least one preset object classification label and linkage range, with one linkage range corresponding to one preset object classification label.

[0114] Obtain the associated range corresponding to the preset object category label that is the same as the object category label;

[0115] Based on the linkage range, the image features that have not yet been input into the primitive comparator are reordered, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator.

[0116] In this embodiment, the linkage range includes first-layer linkage information, which includes at least one pixel position information and a sequence number, with one sequence number corresponding to one pixel position information.

[0117] Based on the linkage range, the image features that have not yet been input into the primitive comparator are reordered, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator, including:

[0118] Determine whether the individual image containing the pixel position information in the first layer of linkage information has already been input into the primitive comparator. If not, then...

[0119] Retrieve single-unit images that have not been input into a primitive comparator;

[0120] For each individual image that has not been input into a primitive comparator, perform the following operations:

[0121] The index corresponding to the pixel position information of a single image that has not been input into the primitive comparator is obtained as the new index of the image feature of that single image.

[0122] In this embodiment, the linkage range includes second-layer linkage information, and the second-layer linkage information includes multiple sequence numbers;

[0123] Obtain image features of other single images besides the individual images that have been input into the primitive comparator and the single images corresponding to the image features with the new sequence number;

[0124] The second layer of linked information is used to randomly assign new serial numbers to the image features of each individual image other than those already input to the primitive comparator and the individual images corresponding to the image features that have obtained new serial numbers.

[0125] by Figure 3 For example, firstly, obtain the single image corresponding to the image features of the change label, that is, the fifteenth single image;

[0126] The position of a single image within the image to be detected is obtained. This position can be obtained during the segmentation of the image to be detected. It can be understood that this position refers to its relative position with other single images. For example, the segmentation is performed in a certain order. Therefore, the left side of the fifteenth single image must be the fourteenth single image, and the top side must be the eleventh single image.

[0127] Obtain the object classification label of the individual image. In this embodiment, the object classification label of the fifteenth individual image is "bed".

[0128] Obtain a preset linkage database, which includes at least one preset object category label and linkage range. One linkage range corresponds to one preset object category label. For example, the preset object category label includes "bed". The linkage range includes first-layer linkage information and second-layer linkage information. The first-layer linkage information includes at least one pixel position information and a sequence number. One sequence number corresponds to one pixel position information.

[0129] In this embodiment, the first layer of linked information for the classification label "bed" includes four pixel position information, which are (A1, B1), (A2, B2), (A3, B3), and (A4, B4).

[0130] Among them, (A1, B1) are located in the eleventh individual image, (A2, B2) are located in the twelfth individual image, (A3, B3) are located in the fifteenth individual image, and (A4, B4) are located in the sixteenth individual image.

[0131] In this embodiment, the pixel position information is preset. Specifically, since the image to be detected is known, the user can clearly know how many individual images each object occupies in the image to be detected, for example... Figure 3 The bed occupies 4 individual images, so the pixel position information of these four individual images is pre-set in the first layer of linkage information.

[0132] In this embodiment, one sequence number corresponds to one pixel position information. For example, (A1, B1) corresponds to sequence number 101, (A2, B2) corresponds to sequence number 102, (A3, B3) corresponds to sequence number 103, and (A4, B4) corresponds to sequence number 104.

[0133] Based on the linkage range, the image features that have not yet been input into the primitive comparator are reordered, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator, including:

[0134] Determine whether the individual image containing the pixel position information in the first layer of linkage information has already been input into the primitive comparator. If not, then...

[0135] Acquire the individual images that have not been input into the primitive comparator (for example, if the image features of the sixteenth individual image and the fifteenth individual image have been input into the primitive comparator, then it is not necessary to acquire them again; only acquire the image features of the eleventh individual image and the twelfth individual image).

[0136] For each individual image that has not been input into a primitive comparator, perform the following operations:

[0137] The index corresponding to the pixel position information of the single image that has not been input into the primitive comparator is obtained as the new index of the image feature of the single image (the index of the image feature of the eleventh single image is 101, and the index of the image feature of the twelfth single image is 102).

[0138] After assigning a new serial number, repeat steps 6 to 7 until the number of obtained change labels exceeds a preset value or all image features are input into the primitive comparator.

[0139] For example, the image features of the eleventh single image are numbered 101 and the image features of the twelfth single image are numbered 102, and the image features are input into the primitive comparator in ascending order of their numbers, so as to obtain the labels output by the primitive comparator, and the labels include change labels.

[0140] It is understood that the scope of this application also includes second-level linkage information, which includes multiple serial numbers;

[0141] Acquire image features of other individual images (the remaining individual images out of the 16 individual images) besides the individual images that have been input to the primitive comparator (the image features of the sixteenth individual image and the image features of the fifteenth individual image have both been input to the primitive comparator) and the individual images corresponding to the image features with new sequence numbers (the sequence number of the image feature of the eleventh individual image is 101, and the sequence number of the image feature of the twelfth individual image is 102).

[0142] The second layer of linked information is used to randomly assign new serial numbers to the image features of each individual image other than those already input to the primitive comparator and the individual images corresponding to the image features that have obtained new serial numbers.

[0143] It is understood that this application can assign different sequence number groups to different labels, and can assign values ​​according to the possibility that the label may change. For example, in one embodiment, the object classification labels include water surface labels and road labels.

[0144] The smallest sequence number in the sequence number group corresponding to the road label is greater than the largest sequence number in the sequence number group corresponding to the water surface label.

[0145] Object classification labels further include building labels;

[0146] The smallest sequence number in the sequence number group corresponding to the water surface label is greater than the largest sequence number in the sequence number group corresponding to the building label.

[0147] Object classification labels further include tree branch labels;

[0148] The smallest sequence number in the sequence number group corresponding to the building label is greater than the largest sequence number in the sequence number group corresponding to the tree branch label.

[0149] In this embodiment, the sequence numbers in the second-layer linkage information are all greater than those in the first-layer linkage information. For example, the sequence numbers of the second-layer linkage information are (1000 to 1050).

[0150] In this embodiment, the method of this application stops when the number of acquired change tags exceeds a preset value. For example, assuming the preset value is 3, it means that there are 3 individual images that have changed. At this time, it is considered that the image to be detected has indeed changed significantly, and other subsequent steps can continue, such as generating alarm information.

[0151] If none of the image features input to the primitive comparator exceed the preset value, it indicates that the image to be detected has not changed much or has not changed at all.

[0152] In this embodiment, the primitive comparator uses ResNet as its skeleton and has 6 input channels. In this embodiment, the model input window is set to 256x256.

[0153] In this embodiment, the trained model becomes a primitive comparator, outputting data between 0 and 1 to represent the similarity of images.

[0154] When in use, the image features are input into the primitive comparator to form the result matrix (i.e., labels); the result matrix is ​​binarized, and in the processed matrix, 0 represents the unchanged area and 1 represents the changed area (i.e., change labels).

[0155] The edge-cloud collaborative image change detection method in this application considers hotspot regions during the initial indexing process, for example, using... Figure 3For example, beds and chairs are usually areas where moving objects are more likely to appear (for example, people are more likely to lie on a bed or sit on a chair), while the walls are less likely to have moving objects. Therefore, when numbering, beds and chairs are considered as hotspot areas, so that hotspot areas can be checked first. If changes are found when checking hotspot areas, other areas can be skipped, thus reducing computing power.

[0156] In the second sorting, the category is taken into consideration. For example, if a change is in a bed, then other individual images with beds will usually also show changes. Through this synergistic effect, it is easier to find the location of the change.

[0157] Compared to the existing method of detecting images one by one with a fixed step size, this application has a faster detection speed and can further save computing power.

[0158] In particular, this application transfers a portion of the computation to the central cloud server, thereby saving the computational burden on the edge servers.

[0159] This application also provides an edge-cloud collaborative image change detection system, which includes a central cloud server and an edge server. The central cloud server and the edge server cooperate to implement the edge-cloud collaborative image change detection method as described above.

[0160] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for image change detection using edge-cloud collaboration, characterized in that, The edge-cloud collaborative image change detection method includes: Step 1: The edge server acquires the image to be detected and sends it to the central cloud server; Step 2: The central cloud server segments the image to be detected into multiple individual images; Step 3: The central cloud server acquires the image features of each individual image; Step 4: The central cloud server sorts the image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server; Step 5: The edge server obtains the trained primitive comparator; Step 6: The edge server inputs each image feature into the primitive comparator in ascending order of sequence number, thereby obtaining the labels output by the primitive comparator, including change labels; Step 7: After the primitive comparator generates the change label, the edge server reorders each image feature that has not yet been input into the primitive comparator according to the second order generation logic, thereby assigning a new sequence number to each image feature that has not yet been input into the primitive comparator. Step 8: Repeat steps 6 to 7 until the number of obtained change labels exceeds a preset value or all image features are input into the primitive comparator.

2. The image change detection method for edge-cloud collaboration as described in claim 1, characterized in that, The central cloud server sorts the image features according to the first order generation logic, thereby assigning a sequence number to each image feature and sending each image feature to the edge server, including: The central cloud server obtains the trained object classifier; The central cloud server sequentially inputs the image features of each individual image into the object classifier to obtain object classification labels; Obtain a first sequence database, which includes at least one preset object category label and a sequence number group. One sequence number group corresponds to one preset object category label. One sequence number group includes multiple different sequence numbers. The sequence numbers in each sequence number group are different. The following operations are performed on each individual image in the order they enter the object classifier: Obtain object classification labels based on image features of individual images; Extract a sequence number from the sequence number group corresponding to the preset object classification label that is the same as the object classification label, and use it as the sequence number of the image feature of the single image.

3. The image change detection method for edge-cloud collaboration as described in claim 2, characterized in that, The step of retrieving a sequence number from the sequence number group corresponding to the preset object classification label that is the same as the object classification label as the sequence number of the image feature of the single image includes: Take the largest sequence number from the sequence number group and use it as the sequence number of the image feature of the single image.

4. The image change detection method for edge-cloud collaboration as described in claim 3, characterized in that, The object classification labels include water surface labels and road labels; The smallest sequence number in the sequence number group corresponding to the road label is greater than the largest sequence number in the sequence number group corresponding to the water surface label.

5. The image change detection method for edge-cloud collaboration as described in claim 4, characterized in that, The image change detection method further includes: The object classification labels further include building labels; The smallest sequence number in the sequence number group corresponding to the water surface label is greater than the largest sequence number in the sequence number group corresponding to the building label.

6. The image change detection method as described in claim 5, characterized in that, The image change detection method further includes: The object classification label further includes a tree branch label; The smallest sequence number in the sequence number group corresponding to the building label is greater than the largest sequence number in the sequence number group corresponding to the tree branch label.

7. The image change detection method for edge-cloud collaboration as described in claim 6, characterized in that, After the primitive comparator generates the change label, the edge server reorders the image features that have not yet been input into the primitive comparator according to the second order generation logic, thereby assigning new sequence numbers to each image feature that has not yet been input into the primitive comparator, including: Obtain the individual image corresponding to the image features of the change label; Obtain the position of the individual image within the image to be detected; Obtain object classification labels from individual images; Obtain a preset linkage database, which includes at least one preset object classification label and linkage range, with one linkage range corresponding to one preset object classification label. Obtain the associated range corresponding to the preset object category label that is the same as the object category label; Based on the linkage range, the image features that have not yet been input into the primitive comparator are reordered, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator.

8. The image change detection method for edge-cloud collaboration as described in claim 7, characterized in that, The linkage range includes a first layer of linkage information, which includes at least one pixel position information and a sequence number, with one sequence number corresponding to one pixel position information. The step of reordering the image features that have not yet been input into the primitive comparator according to the linkage range, thereby assigning new serial numbers to each image feature that has not yet been input into the primitive comparator, includes: Determine whether the individual image containing the pixel position information in the first layer of linkage information has already been input into the primitive comparator. If not, then... Retrieve single-unit images that have not been input into a primitive comparator; For each individual image that has not been input into a primitive comparator, perform the following operations: The index corresponding to the pixel position information of a single image that has not been input into the primitive comparator is obtained as the new index of the image feature of that single image.

9. The image change detection method for edge-cloud collaboration as described in claim 8, characterized in that, The linkage range includes second-layer linkage information, which includes multiple sequence numbers. Obtain image features of other single images besides the individual images that have been input into the primitive comparator and the single images corresponding to the image features with the new sequence number; The second layer of linked information is used to randomly assign new serial numbers to the image features of each individual image other than those already input to the primitive comparator and the individual images corresponding to the image features that have obtained new serial numbers.

10. An edge-cloud collaborative image change detection system, characterized in that, The edge-cloud collaborative image change detection system includes a central cloud server and edge servers, wherein the central cloud server and the edge servers cooperate to implement the edge-cloud collaborative image change detection method as described in any one of claims 1 to 9.

Images