Region matching method and device in environment, electronic equipment, computer readable storage medium and computer program product

By clustering and merging environmental data, the accuracy of regional matching in the environment is improved, the problem of inaccurate analysis results caused by position offset is solved, and the correlation and matching accuracy of data points are enhanced.

CN119007274APending Publication Date: 2024-11-22UBTECH ROBOTICS CORP LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411011214.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When the environment data matches the actual object position, position offset problems are prone to occur, resulting in inaccurate analysis results.

Method used

By performing the first clustering process on the point set data, the second clustering region matching the pre-calibrated prior clustering region is determined, and the merge process is performed to obtain the third clustering center of the third clustering region as the initial point Perform the second clustering process to improve matching accuracy.

Benefits of technology

The accuracy of regional matching in the environment is improved, making the data point correlation in the same clustering area stronger, and the matching accuracy between the target clustering area and the prior clustering area is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007274A_ABST
    Figure CN119007274A_ABST
Patent Text Reader

Abstract

The invention provides a region matching method and device in an environment, electronic equipment, a computer program product and a computer readable storage medium. The method comprises the steps of obtaining point set data of a target environment, wherein the point set data comprises N data points; n data points in the point set data are subjected to first clustering processing, M first clustering areas are obtained, each first clustering area corresponds to a first clustering center, each first clustering area comprises P data points, and P is smaller than or equal to N; determining a second clustering region matched with a pre-calibrated prior clustering region from the M first clustering regions; combining the at least one first clustering region with the second clustering region to obtain a third clustering region matched with the prior clustering region; and taking a third clustering center of the third clustering region as a clustering initial point, and performing second clustering processing on the point set data to obtain a first target clustering region matched with the prior clustering region. According to the invention, the accuracy of region matching in the environment can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to point set data processing technology, and more particularly to a method, device, electronic device, computer-readable storage medium and computer program product for region matching in an environment. Background Art

[0002] When matching the environmental data generated by the environment with the actual objects in the environment, due to the hardware accuracy of obtaining the environmental data or the software algorithm during matching, it is easy for the collected environmental data to be offset from the expected position of the actual object, resulting in inaccurate subsequent analysis results. Summary of the invention

[0003] Embodiments of the present application provide a method, device, electronic device, computer-readable storage medium, and computer program product for region matching in an environment, which can improve the accuracy of region matching in an environment.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a method for area matching in an environment, the method comprising: acquiring point set data of a target environment, wherein the point set data comprises N data points; performing a first clustering process on the N data points in the point set data to obtain M first clustering regions, wherein each of the first clustering regions corresponds to a first clustering center, each of the first clustering regions contains P data points, N, M, and P are positive integers, and P≤N; determining a second clustering region that matches a pre-calibrated priori clustering region from the M first clustering regions; merging at least one of the first clustering regions with the second clustering region to obtain a third clustering region that matches the priori clustering region; and performing a second clustering process on the point set data using the third clustering center of the third clustering region as a clustering initial point to obtain a first target clustering region that matches the priori clustering region.

[0006] The present application provides a region matching device in an environment, including:

[0007] An acquisition module, used to acquire point set data of a target environment, wherein the point set data includes N data points;

[0008] A first clustering module, used for performing a first clustering process on the N data points to obtain M first clustering regions, wherein each of the first clustering regions corresponds to a first clustering center, each of the first clustering regions contains P data points, N, M, and P are positive integers, and P≤N;

[0009] A pre-matching module, used to determine a second clustering region matching a pre-calibrated priori clustering region from the M first clustering regions;

[0010] a merging module, configured to merge at least one of the first clustering regions with the second clustering region to obtain a third clustering region matching the priori clustering region;

[0011] The second clustering module is used to perform a second clustering process on the point set data using the third cluster center of the third cluster area as a clustering initial point to obtain a first target cluster area that matches the priori cluster area.

[0012] In the above scheme, the first clustering module is also used to use the prior cluster center of the prior clustering area as the clustering initial point, perform a first clustering process on the N data points in the point set data, and obtain M first clustering areas; or use the number of the prior clustering areas as the number of clustering areas, perform a first clustering process on the N data points in the point set data, and obtain M first clustering areas, where M is equal to the number of the prior clustering areas.

[0013] In the above scheme, the pre-matching module is also used to determine the distance between the prior cluster center of the prior cluster area and each of the first cluster centers; from the M first cluster areas, the first cluster area with the smallest distance is screened out, and the screened out first cluster area with the smallest distance is used as the second cluster area matching the prior cluster area.

[0014] In the above scheme, the merging module is also used to expand the range of the prior clustering area based on the first position deviation threshold corresponding to the target environment to obtain an expanded first prior clustering area; perform cluster center detection on the expanded first prior clustering area to obtain a first detection result; when the first detection result indicates that the expanded first prior clustering area includes at least one other cluster center and a second cluster center, merge the first clustering area corresponding to the at least one other cluster center with the second clustering area to obtain a third clustering area matching the prior clustering area, wherein the other cluster centers are cluster centers among the first cluster centers corresponding to the M first clustering areas, except the second cluster center, and the second cluster center is the center of the second clustering area.

[0015] In the above scheme, the device also includes a third clustering module, which is used to perform a second clustering processing on the point set data with the second cluster center of the second cluster area as the clustering initial point when the first detection result indicates that the expanded first prior clustering area does not include the other cluster centers, so as to obtain a second target clustering area matching the prior clustering area.

[0016] In the above solution, the device also includes a matching cancellation module, which is used to cancel the matching relationship between the second clustering area and the priori clustering area when the first detection result indicates that the second cluster center of the second clustering area is outside the range of the expanded first priori clustering area.

[0017] In the above scheme, the second clustering module is also used to obtain the intermediate clustering area obtained by performing the second clustering processing on the point set data after the second clustering processing is performed on the point set data; based on the second position deviation threshold corresponding to the target environment, the range of the prior clustering area is expanded to obtain the expanded second prior clustering area; cluster center detection is performed on the expanded second prior clustering area to obtain a second detection result; when the second detection result indicates that the expanded second prior clustering area includes intermediate cluster centers of multiple intermediate clustering areas, the multiple intermediate clustering areas are merged to obtain a merged clustering area; wherein the second position deviation threshold is greater than the first position deviation threshold, and the first position deviation threshold is the position deviation threshold used when merging at least one of the first clustering areas with the second clustering area; based on the merged clustering area, a first target clustering area matching the prior clustering area is determined.

[0018] In the above scheme, the second clustering module is also used to use the fourth cluster center of the merged cluster area as the clustering initial point, perform iterative clustering processing on the point set data, and obtain the first target clustering area matching the prior clustering area; wherein the iterative clustering termination condition of the iterative clustering processing includes at least one of the following: the number of clustering times reaches a preset number, the repetition rate of the data points included in the cluster area obtained by this clustering and the data points included in the cluster area obtained by the previous clustering reaches a preset repetition rate, and the position deviation between the cluster center of the cluster area obtained by this clustering and the cluster center of the cluster area obtained by the previous clustering is less than a preset deviation.

[0019] In the above scheme, the acquisition module is also used to acquire initial point set data generated for the target environment, wherein the initial point set data includes N initial data points; and position smoothing processing is performed on the N initial data points in the initial point set data to obtain the point set data.

[0020] In the above scheme, the acquisition module is also used to sort the N initial data points based on the timing information of each of the initial data points; for each of the initial data points, perform the following processing: based on the sorting result of the sorting, determine the R surrounding data points associated with the initial data point, and the timing difference between the surrounding data points and the initial data point is less than a preset threshold; determine n surrounding data points that have completed position smoothing and m surrounding data points that have not completed position smoothing among the R surrounding data points, where R, m, and n are all integers, and m=Rn; obtain the first initial position information of the initial data point, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points; based on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points, perform position smoothing processing on the initial data point to obtain the second updated position information of the initial data point; use the second updated position information of each of the initial data points in the initial point set data as the position information of each data point in the point set data.

[0021] In the above scheme, the acquisition module is also used to determine, based on the sorting result, R surrounding data points adjacent to and before the initial data point when the timing sequence number of the initial data point is greater than the first designated sequence number; when the timing sequence number of the initial data point is greater than the second designated sequence number and less than or equal to the first designated sequence number, based on the sorting result, n surrounding data points adjacent to and before the initial data point, and m surrounding data points adjacent to and after the initial data point; when the timing sequence number of the initial data point is equal to the second designated sequence number, based on the sorting result, R surrounding data points adjacent to and after the initial data point are determined.

[0022] In the above scheme, the acquisition module is also used to obtain the first weight corresponding to the first initial position information, the second weight corresponding to the second initial position information of the m surrounding data points, and the third weight corresponding to the first updated position information of the n surrounding data points, wherein the first weight is greater than the second weight, and the third weight is greater than the second weight; based on the first weight, the second weight and the third weight, the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points are weightedly summed to obtain the second updated position information of the initial data points.

[0023] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0024] A memory for storing computer executable instructions;

[0025] The processor is used to implement the area matching method in the environment provided by the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0026] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing a region matching method in an environment provided by an embodiment of the present application when executed by a processor.

[0027] An embodiment of the present application provides a computer program product, including a computer program or a computer executable instruction. When the computer program or the computer executable instruction is executed by a processor, the area matching method in the environment provided by the embodiment of the present application is implemented.

[0028] Applying the above-mentioned embodiment, the point set data of the target environment is subjected to a first clustering process to obtain M first clustering regions, and the corresponding second clustering regions are pre-matched from the M first clustering regions for the prior clustering regions pre-calibrated for the target environment, and the second clustering regions and at least one first clustering region are merged to obtain a third clustering region that matches each prior clustering region, and then, the point set data is subjected to a second clustering process using the third clustering center of the third clustering region as the clustering initial point, thereby obtaining a first target clustering region that matches the prior clustering region; since the embodiment of the present application performs two clustering processes on the point set data, and the second clustering is performed based on the pre-matching and merging of the clustering regions after the first clustering, the second clustering can obtain a more accurate clustering result, so that the data points in the same clustering region are more strongly correlated, which can improve the clustering accuracy of the first target clustering region, thereby improving the matching accuracy between the first target clustering region and the prior clustering region. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a structural diagram of the regional matching system architecture in the environment provided by the embodiment of the present application;

[0030] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0031] Figure 3A It is a flowchart of a region matching method in an environment provided by an embodiment of the present application;

[0032] Figure 3B is a flowchart of a pre-matching method provided in an embodiment of the present application;

[0033] Figure 3C It is a schematic diagram of the process of the cluster region merging method provided in an embodiment of the present application;

[0034] Figure 4A is a schematic diagram of an image to be observed in a target environment provided by an embodiment of the present application;

[0035] Figure 4B is a schematic diagram of a priori clustering area corresponding to the calibration of the image to be observed provided in an embodiment of the present application;

[0036] Figure 5A is a schematic diagram of initial eye movement data generated for an image to be observed provided by an embodiment of the present application;

[0037] Figure 5B is a schematic diagram of eye movement data after position smoothing provided in an embodiment of the present application;

[0038] Fig. 6A is a first schematic diagram of clustering areas and clustering centers provided in an embodiment of the present application;

[0039] Figure 6B is a first schematic diagram of an expanded priori clustering region provided in an embodiment of the present application;

[0040] Fig. 7A is a first schematic diagram of clustering areas and clustering centers provided in an embodiment of the present application;

[0041] Figure 7B is a second schematic diagram of the expanded priori clustering area provided in an embodiment of the present application;

[0042] Figure 8 is a schematic diagram of a target clustering area provided in an embodiment of the present application;

[0043] Fig. 9 It is a scene diagram of the target environment provided in an embodiment of the present application.

[0044] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0046] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0047] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0048] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0049] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0050] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0051] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0052] 1) Environmental data, i.e., data used to describe the environment. For example, environmental data can be eye movement data that describes the environment from the perspective of the observation position, or two-dimensional image data that describes the environment from the perspective of the color of the pixel points (the grayscale value of the RGB channel), or three-dimensional image data that describes the environment from the perspective of combining color and pixel point positions. The three-dimensional image data can be point cloud data. Eye movement data, two-dimensional image data, and three-dimensional image data can all be regarded as point set data. The point set data corresponding to eye movement data and two-dimensional image data is in two-dimensional space, while the point cloud data corresponding to three-dimensional image data is in three-dimensional space.

[0053] 2) Eye movement data is a type of data collected through eye movement detection technology, which can reflect an individual's visual attention, gaze behavior and cognitive process. Eye movement detection technology is achieved by detecting and recording data such as eye movement trajectory, gaze point, gaze time, eye movement path, etc.

[0054] Eye movement data at least includes gaze points: gaze points record the position where the subject is looking on the observation plane, usually expressed in pixel coordinates on the screen. It reflects the focus position of the subject's line of sight and can be used to analyze the distribution of the subject's visual attention.

[0055] 3) Clustering is an unsupervised learning method mainly used for data mining and statistical data analysis. Its basic goal is to classify a set of data, but unlike traditional classification methods, cluster analysis is performed without knowing the category to which the data belongs. The result of clustering is to divide the data into several clusters (i.e., clustering areas in this application), so that the data points in the same cluster are more similar to each other, while the data points in different clusters are less similar.

[0056] When matching the positions of environmental data generated by the environment with actual objects in the environment, it is easy for the collected environmental data to be offset from the expected position of the actual object due to the hardware accuracy of obtaining the environmental data or the software algorithm during matching. For example, when using an eye tracker to obtain the viewing area and position of the human eye, if the eye tracker hardware and software algorithm are not well calibrated, the human eye may actually be looking at area A, but the data obtained by the eye tracker will indicate that the human eye is looking at area B, which will lead to inaccurate subsequent analysis results.

[0057] Embodiments of the present application provide a method, apparatus, device, computer-readable storage medium, and computer program product for region matching in an environment, which can improve the accuracy of region matching in an environment.

[0058] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the regional matching system 100 in the environment provided by the embodiment of the present application. In order to support the regional matching application in an environment, the terminal 401 is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0059] The terminal 401 is used for sending a request for area matching in the environment to the server 200 in response to an instruction for area matching in the environment.

[0060] When the server 200 receives a request for area matching in the environment, it responds to the request for area matching in the environment, obtains point set data of the target environment, performs a first clustering process on N data points in the point set data, and obtains M first clustering regions, wherein each first clustering region corresponds to a first clustering center, each first clustering region contains P data points, N, M, and P are positive integers, and P≤N; determines a second clustering region that matches a pre-calibrated prior clustering region from the M first clustering regions; merges at least one first clustering region with the second clustering region to obtain a third clustering region that matches the prior clustering region; uses the third clustering center of the third clustering region as a clustering initial point, performs a second clustering process on the point set data, obtains a first target clustering region that matches the prior clustering region, determines an area matching result for the target environment, and sends the area matching result to the terminal 401.

[0061] In some embodiments, the area matching method in the environment provided by the embodiments of the present application can be implemented by various electronic devices, for example, it can be implemented by the terminal 401 alone, it can also be implemented by the server 200 alone, or it can be implemented by the terminal 401 and the server 200 in collaboration.

[0062] In some embodiments, the terminal 401 can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, a car terminal, etc.

[0063] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0064] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present application, Figure 2The electronic device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0065] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0067] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0068] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0069] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0070] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0071] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0072] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0073] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0074] In some embodiments, the area matching device in the environment provided by the embodiments of the present application can be implemented in a software manner. Figure 2 The area matching device 455 in the environment stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 4551, a first clustering module 4552, a pre-matching module 4553, a merging module 4554, and a second clustering module 4555. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0075] In other embodiments, the region matching device in the environment provided by the embodiments of the present application can be implemented in hardware. As an example, the region matching device in the environment provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the region matching method in the environment provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0076] The following will describe the region matching method in the environment provided by the embodiment of the present application in conjunction with the accompanying drawings. As mentioned above, the electronic device that implements the region matching method in the environment of the embodiment of the present application may be the terminal 401, the server 200, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0077] The regional matching method in the environment of the embodiment of the present application is described by taking the execution subject as the server 200 as an example. Figure 3A , Figure 3A is a first flow chart of the region matching method in the environment provided by the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0078] In step 101, point set data of a target environment is obtained.

[0079] The point set data includes N data points.

[0080] In the embodiment of the present application, when the point set data is represented as eye movement data or three-dimensional image data, the data points in the point set data have corresponding position information and timing information. When the point set data is represented as eye movement data, the position information of the data points can be two-dimensional coordinates; when the point set data is represented as three-dimensional image data, the position information of the data points can be three-dimensional coordinates. When the point set data is represented as two-dimensional image data, the data points can have pixel information or depth information.

[0081] As an example, when the target environment is represented as the image information or text information on the electronic device screen observed by the observed object from the observation position, the point set data of the target environment can be eye movement data collected by an eye tracker when the observed object observes the screen from the observation position.

[0082] In some embodiments, Figure 3A The step 101 shown can be implemented in the following manner.

[0083] Initial point set data generated for a target environment is obtained, wherein the initial point set data includes N initial data points; and position smoothing processing is performed on the N initial data points in the initial point set data to obtain point set data.

[0084] When the initial point set data (such as initial eye movement data) for the target environment is collected by hardware equipment (such as an eye tracker), the initial point set data often has a certain degree of jitter due to equipment accuracy issues. Therefore, after obtaining the initial point set data generated for the target environment, the data points in the initial point set data can be first smoothed, that is, the initial point set data can be de-jittered.

[0085] In actual implementation, position smoothing processing may be performed on each initial data point based on the time sequence of the N initial data points in the initial point set data, thereby obtaining point set data after position smoothing.

[0086] In some embodiments, the above “performing position smoothing processing on N initial data points in the initial point set data to obtain point set data” can be implemented in the following manner.

[0087] Based on the timing information of each initial data point, N initial data points are sorted; for each initial data point, the following processing is performed: based on the sorting result, R surrounding data points associated with the initial data point are determined, and the timing difference between the surrounding data points and the initial data point is less than a preset threshold; n surrounding data points that have completed position smoothing and m surrounding data points that have not completed position smoothing among the R surrounding data points are determined, wherein R, m, and n are all integers, and m=Rn; the first initial position information of the initial data point, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points are obtained; based on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points, the initial data point is positionally smoothed to obtain the second updated position information of the initial data point; the second updated position information of each initial data point in the initial point set data is used as the position information of each data point in the point set data.

[0088] Among them, the time series difference between the surrounding data points and the initial data point is less than the preset threshold. In actual implementation, the preset threshold can be set based on the actual smoothing requirements. If more surrounding data points are needed to perform position smoothing on the current initial data point in actual implementation, the value of the preset threshold can be set relatively large. On the contrary, if more surrounding data are not needed to perform position smoothing on the current initial data point, the value of the preset threshold can be set relatively small.

[0089] As an example, if 5 surrounding data points are needed to perform position smoothing on the initial data point, the preset threshold can be set to 6. If the initial point set data includes 100 initial data points, for the initial data point with a time sequence number of 10, any 5 data points with time sequence numbers of 5-15 can be used as the surrounding data points of the initial data point with a time sequence number of 10.

[0090] In some embodiments, determining the R surrounding data points associated with the initial data point based on the sorting result may include the following cases: when the timing sequence number of the initial data point is greater than the first specified sequence number, based on the sorting result, determining the R surrounding data points adjacent to and before the initial data point; when the timing sequence number of the initial data point is greater than the second specified sequence number and less than or equal to the first specified sequence number, based on the sorting result, determining n surrounding data points adjacent to and before the initial data point, and m surrounding data points adjacent to and after the initial data point; when the timing sequence number of the initial data point is equal to the second specified sequence number, based on the sorting result, determining the R surrounding data points adjacent to and after the initial data point.

[0091] In actual implementation, the position smoothing process can be performed on each initial data point in the order of the time series number from small to large based on the sorting result. Since the position information of the initial data point after the position smoothing process is more consistent with the real position, when selecting R surrounding data points for the current initial data point, the surrounding data points with the time series number before the current initial data point can be used as much as possible. However, when the time series of the initial data point is relatively early, that is, the time series number of the initial data point is relatively small, the surrounding data points before the initial data point may be less than R. Therefore, in this case, the surrounding data points after the initial data point can also be selected. Accordingly, the embodiment of the present application sets a first designated sequence number and a second designated sequence number, and determines the determination method of the R surrounding data points of the initial data point according to the size relationship between the time series number of the initial data point and the first designated sequence number and the second designated sequence number. Among them, the second designated sequence number is usually the minimum value of the time series number of the N initial data points, and the first designated sequence number can be set according to the above-mentioned preset threshold.

[0092] As an example, if three surrounding data points are needed to perform position smoothing on the initial data point, the preset threshold can be set to 4, that is, the difference between the time series numbers of the surrounding data points and the time series number of the initial data point should be less than 4. In this regard, if the time series numbers are consecutive positive integers starting from 1 after the N initial data points are sorted in time series, the first designated number can be set to 3, and the second designated number can be set to 1. According to the size relationship between the time sequence number of the initial data point and the first designated sequence number and the second designated sequence number, the following situations can be included: when the time sequence number of the initial data point is greater than 3, based on the sorting result, 3 surrounding data points adjacent to the initial data point and located before the initial data point can be determined; when the time sequence number of the initial data point is greater than 1 and less than or equal to 3, all surrounding data points adjacent to the initial data point and located before the initial data point (that is, n surrounding data points, and these n surrounding data points have completed position smoothing) and m surrounding data points adjacent to the initial data point and located after the initial data point (m=3-n, these m surrounding data points have not yet been position smoothed) can be determined first; when the time sequence number of the initial data point is equal to 1, 3 surrounding data points adjacent to the initial data point and located after the initial data point can be determined.

[0093] As an example, when the time sequence number of the initial data point is 6, the three surrounding data points associated with the initial data point can be determined as the three surrounding data points with time sequence number 5, time sequence number 4 and time sequence number 3; when the time sequence number of the initial data point is 3, first determine the two surrounding data points before the initial data point (that is, the surrounding data points corresponding to time sequence number 2 and the surrounding data points corresponding to time sequence number 1), and then, after determining the one surrounding data point after the initial data point (that is, the surrounding data point with time sequence number 4), the three surrounding data points with time sequence number 1, time sequence number 2 and time sequence number 4 are used as the surrounding data points associated with the initial data point; when the time sequence number of the initial data point is 1, the three surrounding data points with time sequence number 2, time sequence number 3 and time sequence number 4 after the initial data point can be determined as the surrounding data points associated with the initial data point.

[0094] After determining the R surrounding data points associated with the current initial data point, determine the n surrounding data points that have completed position smoothing and the m surrounding data points that have not completed position smoothing among the R surrounding data points, and use the first updated position information of the n surrounding data points after completing position smoothing, the second initial position information corresponding to the m surrounding data points, and the first initial position information corresponding to the current initial data point to perform position smoothing processing on the current initial data point.

[0095] In some embodiments, the above-mentioned "based on the first initial position information, the second initial position information of m surrounding data points and the first updated position information of n surrounding data points, position smoothing processing is performed on the initial data points to obtain the second updated position information of the initial data points" can be achieved in the following way.

[0096] Obtain a first weight corresponding to the first initial position information, a second weight corresponding to the second initial position information of the m surrounding data points, and a third weight corresponding to the first updated position information of the n surrounding data points, wherein the first weight is greater than the second weight, and the third weight is greater than the second weight; based on the first weight, the second weight and the third weight, perform weighted sum processing on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points to obtain the second updated position information of the initial data points.

[0097] In actual implementation, when performing position smoothing calculation on the initial data points, different weights can be assigned to different data to improve the accuracy of position smoothing. Specifically, the weight corresponding to the first initial position information of the initial data point before position smoothing is the largest, the weight corresponding to the first updated position information of the n surrounding data points after position smoothing is greater than the weight corresponding to the second initial position information of the m surrounding data points before position smoothing among the R related initial data points, and the weight corresponding to the first updated position information of the surrounding data points whose timing is closer to the initial data point among the n surrounding data points is greater, and the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points are assigned corresponding weights, the first initial position information is assigned a first weight, the second initial position information of the m surrounding data points is assigned a second weight, and the first updated position information of the n surrounding data points is assigned a third weight, the first weight is greater than the second weight, and the third weight is greater than the second weight.

[0098] As an example, when position smoothing is performed on an initial data point with a time sequence number of 2, it is determined that the three surrounding data points associated with the initial data point are the surrounding data point with a time sequence number of 1, the surrounding data point with a time sequence number of 3, and the surrounding data point with a time sequence number of 4, wherein the surrounding data point with a time sequence number of 1 has completed position smoothing, while the two surrounding data points with time sequence numbers of 3 and 4 have not completed position smoothing. The first initial position information corresponding to the initial data point, the second initial position information of the surrounding data point with a time sequence number of 3, the second initial position information of the surrounding data point with a time sequence number of 4, and the first updated position information corresponding to the surrounding data point with a time sequence number of 1 are obtained. Then, the first initial position information of the initial data point is assigned a maximum weight of 0.4 (first weight), and the first updated position information of the surrounding data point with the time sequence number 1 is assigned a weight of 0.3 (third weight). Since the surrounding data point with the time sequence number 3 is closer to the initial data point than the surrounding data point with the time sequence number 4, the second initial position information of the surrounding data point with the time sequence number 3 is assigned a weight of 0.2, and the second initial position information of the surrounding data point with the time sequence number 4 is assigned a weight of 0.1. The weights corresponding to the two second initial position information are the second weights. Overall, the first weight is greater than the second weight, and the third weight is greater than the second weight.

[0099] In some embodiments, the above-mentioned "based on the first weight, the second weight and the third weight, the first initial position information, the second initial position information of m surrounding data points and the first updated position information of n surrounding data points are weighted summed to obtain the second updated position information of the initial data point" can be implemented in the following way.

[0100] Determine a first product value between the first initial position information and the first weight, a second product value between the second initial position information and the corresponding second weight, and a third product value between the first updated position information and the corresponding third weight; determine the second updated position information after the initial data point completes position smoothing based on the sum of the first product value, the second product value, and the third product value.

[0101] As an example, the initial point set data includes N=100 initial data points. After the 100 initial data points are sorted according to the time series information, the time series numbers of the sorted initial data points are 1-100. The preset threshold is set to 4, the first designated number is 3, and the second designated number is 1. It is necessary to use 3 surrounding data points to perform position smoothing processing on the initial data points. The position smoothing processing can be performed on the initial data points in the order of small to large time series numbers.

[0102] For the initial data point with time sequence number n=1 (i.e., the initial data point of the first frame), the first initial position information of the initial data point of the first frame before updating (i.e., before position smoothing), the second initial position information of the initial data point of the second frame (i.e., the surrounding data point with time sequence number 2) before updating, the second initial position information of the initial data point of the third frame (i.e., the surrounding data point with time sequence number 3) before updating, and the second initial position information of the initial data point of the fourth frame (i.e., the surrounding data point with time sequence number 4) before updating are used to perform weighted summation, and future data is used to perform position smoothing on the initial data point of the first frame. Specifically, the second updated position information of the initial data point of the first frame can be calculated using the following formula:

[0103]

[0104] In the formula, P represents an initial data point. The lower right subscript of P indicates the time series number of this initial data point, and the upper right subscript of P indicates whether to use the data before or after the update (i.e., the data before or after position smoothing). If it is marked with new, it indicates the updated data, and if it is marked with old, it indicates the data before the update. Indicates the second updated position information of the initial data point with the time sequence number 1, represents the first initial position information of the initial data point with the time series number 1 (the corresponding first weight is 0.4), represents the second initial position information of the initial data point with the time series number of 2 (the corresponding second weight is 0.3), represents the second initial position information of the initial data point with the time series number of 3 (the corresponding second weight is 0.2), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0105] For the initial data point with time sequence number n=2 (i.e., the initial data point of the second frame), the second updated position information of the initial data point of the second frame after updating is weighted summed using the first initial position information of the initial data point of the second frame before updating, the first updated position information of the initial data point of the first frame after updating, the second initial position information of the initial data point of the third frame before updating, and the second initial position information of the initial data point of the fourth frame before updating. The position of the initial data point of the second frame is smoothed using future data and previous data. Specifically, the second updated position information of the initial data point of the second frame can be calculated using the following formula:

[0106]

[0107] in, represents the second updated position information of the initial data point with the time sequence number 2, represents the first updated position information of the initial data point with the time sequence number 1 (the corresponding third weight is 0.3), represents the first initial position information of the initial data point with the time series number of 2 (the corresponding first weight is 0.4), represents the second initial position information of the initial data point with the time series number of 3 (the corresponding second weight is 0.2), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0108] For the initial data point with time sequence number n=3 (i.e., the initial data point of the third frame), the second updated position information of the initial data point of the third frame after updating is weighted summed using the first updated position information of the initial data point of the first frame after updating, the first updated position information of the initial data point of the second frame after updating, the first initial position information of the initial data point of the third frame before updating, and the second initial position information of the initial data point of the fourth frame before updating. The position of the initial data point of the third frame is smoothed using future data and previous data. Specifically, the second updated position information of the initial data point of the third frame can be calculated using the following formula:

[0109]

[0110] in, represents the second updated position information of the initial data point with the time sequence number 3, represents the first updated position information of the initial data point with the time sequence number 1 (the corresponding third weight is 0.2), represents the first updated position information of the initial data point with the time sequence number of 2 (the corresponding third weight is 0.3), represents the first initial position information of the initial data point with the time series number of 3 (the corresponding first weight is 0.4), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0111] For the initial data point with a time sequence number greater than 3 (n>3), for example, the initial data point with a time sequence number n=24, the second updated position information of the initial data point of the 24th frame after the update uses the first updated position information of the initial data point of the 21st frame after the update, the first updated position information of the initial data point of the 22nd frame after the update, the first updated position information of the initial data point of the 23rd frame after the update, and the first initial position information of the initial data point of the 24th frame before the update, and performs weighted summation, and uses the previous data to perform position smoothing on the initial data point of the 24th frame. Specifically, the second updated position information of the initial data point of the 24th frame can be calculated using the following formula:

[0112]

[0113] in, represents the second updated position information of the initial data point with the time sequence number of 24, represents the first updated position information of the initial data point with the time sequence number 21 (the corresponding third weight is 0.1), represents the first updated position information of the initial data point with the time sequence number of 22 (the corresponding third weight is 0.2), represents the first updated position information of the initial data point with the time sequence number of 23 (the corresponding third weight is 0.3), Represents the first initial position information of the initial data point with the time series number of 24 (the corresponding first weight is 0.4).

[0114] In addition, since the point set data in the embodiment of the present application has position information, the data points are generally represented by coordinate values ​​of (x, y). Therefore, when actually calculating the second updated position information of the initial data point after the update, the relevant data of the x-coordinate will be used to calculate the updated value of the x-coordinate, and the relevant data of the y-coordinate will be used to calculate the updated value of the y-coordinate.

[0115] After calculating and obtaining the second updated position information corresponding to each initial data point in the initial point set data in the above manner, the second updated position information of each initial data point in the initial point set data is used as the position information of each data point in the point set data to obtain the point set data.

[0116] Continue reading Figure 3A , and the description will continue with the above step 101.

[0117] In step 102, a first clustering process is performed on N data points to obtain M first clustering regions.

[0118] Each first clustering region corresponds to a first clustering center, the first clustering center is the center of the first clustering region, each first clustering region contains P data points, N, M, P are positive integers, and P≤N.

[0119] In some embodiments, step 102 can be implemented in the following ways: using the prior cluster center of the prior cluster area as the clustering initial point, performing a first clustering process on the N data points in the point set data, and obtaining M first clustering regions; or using the number of prior clustering regions as the number of clustering regions, performing a first clustering process on the N data points in the point set data, and obtaining M first clustering regions, where M is equal to the number of the prior clustering regions.

[0120] In the embodiment of the present application, the a priori clustering area is an area pre-calibrated for a designated object in the target environment.

[0121] As an example, when the target environment is represented as image information or text information on the screen of an electronic device observed by an observed object from an observation position, and the point set data of the target environment is eye movement data collected by an eye tracker when the observed object observes the screen from the perspective of the observation position, the designated object can be the image information or text information on the screen.

[0122] For example, see Figure 4A , Figure 4A The figure shows an image displayed by the electronic device screen to the viewing object, including designated object 501, designated object 502, designated object 503, and designated object 504. Figure 4A The four specified objects in the image shown are pre-calibrated with the corresponding prior clustering regions, see Figure 4B , the a priori clustering area is represented by a rectangular area, and the positions of the four specified objects are marked by four rectangular areas in the image, that is, the a priori clustering area 5011 corresponds to the specified object 501 in Figure 4A As shown in the image, the prior clustering region 5021 corresponds to the position of the specified object 502 in the image, the prior clustering region 5031 corresponds to the position of the specified object 503 in the image, and the prior clustering region 5041 corresponds to the position of the specified object 504 in the image.

[0123] For example, see Figure 5A , Figure 5A The display is for the observation object Figure 4A The initial eye movement data (i.e., initial point set data) when observing the image shown. Figure 5A It can be seen that the observed object has viewed the designated object 503, but the initial eye movement data obtained is overall offset upward, so there are some eye movement data points located outside the priori clustering area 5031. Figure 5A After smoothing the position of the initial eye movement data points in Figure 5B The smoothed eye movement data (equivalent to point set data) shown in Figure 2 is the same as the smoothed eye movement data (equivalent to point set data). Figure 5A In terms of Figure 5B The jitter of the eye movement data in the video is significantly reduced, and the eye movement trajectory can be clearly seen. Figure 5B It can be seen that the smoothed eye movement data still has a position offset with respect to the priori clustering area 5031. Therefore, in order to match the position of the data points in the eye movement data with the corresponding priori clustering areas, the N data points in the eye movement data can be first clustered to obtain M first clustering areas, each of which includes P eye movement data points. Then, the first clustering areas are matched with the priori clustering areas.

[0124] As an example, since the point set data is clustered in order to match it with the prior clustering area, the prior clustering center of the prior clustering area can be directly used as the clustering initial point of the first clustering processing, and the N data points in the point set data are subjected to the first clustering processing to obtain M first clustering areas, where the number of first clustering areas is usually equal to the number of prior clustering areas.

[0125] As an example, the number of priori clustering regions may be directly used as the number of clustering regions, so that after performing the first clustering process on the N data points in the point set data, the number of first clustering regions equal to the number of priori clustering regions is obtained.

[0126] As an example, it is also possible to randomly cluster the point set data without specifying the number of clustering areas or the clustering initial point for the first clustering process, given the distance conditions required for clustering, to obtain M first clustering areas. In this case, M is a random value.

[0127] In step 103, a second cluster region matching a pre-calibrated priori cluster region is determined from the M first cluster regions.

[0128] In some embodiments, see Figure 3B , step 103 can be implemented by following steps 1031 to 1032:

[0129] In step 1031, the distance between the priori cluster center of the priori cluster area and each first cluster center is determined.

[0130] In step 1032, a first cluster region with the smallest distance is screened out from the M first cluster regions, and the screened out first cluster region with the smallest distance is used as a second cluster region matching the priori cluster region.

[0131] In the embodiment of the present application, the a priori cluster center is the center of the a priori cluster area.

[0132] In actual implementation, the first clustering region can be pre-matched for the priori clustering region according to the distance between the priori clustering center and the first clustering center of the priori clustering region. Specifically, the priori clustering center corresponding to each priori clustering region and the first clustering center corresponding to each first clustering region are determined; the distance between the priori clustering center and the first clustering center is determined, the first clustering center with the smallest distance from the priori clustering center is determined, and the first clustering region corresponding to the first clustering center with the smallest distance is used as the second clustering region matched with the priori clustering center.

[0133] See also Fig. 6A , Fig. 6A It is shown in Figure 5BThe four a priori clustering regions corresponding to the four a priori clustering regions and the a priori clustering centers corresponding to the four a priori clustering regions (i.e., a priori clustering center 1, a priori clustering center 2, a priori clustering center 3, and a priori clustering center 4), Figure 5B The four first clustering areas corresponding to the eye movement data shown and the first clustering centers (i.e., first clustering center A, first clustering center B, first clustering center C, and first clustering center D) corresponding to the four first clustering areas are shown. In order to facilitate the determination of the distance between the first clustering center and the priori clustering center, Fig. 6A No eye movement data are shown.

[0134] As an example, for any a priori clustering region, the distance between the a priori clustering center of the a priori clustering region and each first clustering center can be determined, and the first clustering region corresponding to the first clustering center with the smallest distance is determined as the second clustering region of the a priori clustering region. For example, for the a priori clustering region 5011, the first distance between the a priori clustering center 1 and the first clustering center A, the second distance between the a priori clustering center 1 and the first clustering center B, the third distance between the a priori clustering center 1 and the first clustering center C, and the fourth distance between the a priori clustering center 1 and the first clustering center D can be determined respectively, and the size relationship among the first distance, the second distance, the third distance, and the fourth distance is determined, and the first clustering center A corresponding to the first distance is determined to be the one with the smallest distance from the a priori clustering center 1, and the first clustering region corresponding to the first clustering center A is used as the second clustering region corresponding to the a priori clustering region 5011. Similarly, for the priori clustering region 5021, according to the distances between the priori clustering region and all the first cluster centers, a second clustering region corresponding to the priori clustering region 5021 is determined among the four first clustering regions. The priori clustering regions 5031 and 5041 are similar and will not be described in detail here.

[0135] As an example, after determining the first cluster center corresponding to the first cluster area, the first cluster center can be matched with the prior cluster center, and then the optimal matching combination with the smallest distance between the first cluster center and the prior cluster center in each matching combination is determined, and the second cluster area corresponding to each prior cluster area is determined according to the correspondence between the first cluster center and the prior cluster center in the optimal matching combination. For example, for the prior cluster center 1, there are four first cluster centers that can be matched for it. If the prior cluster center matches the first cluster center A, the first cluster centers that can be matched by the prior cluster center 2 are the other three first cluster centers except the first cluster center A. If the prior cluster center 2 matches the first cluster center B, the first cluster centers that can be matched by the prior cluster center 3 are the other two first cluster centers except the first cluster center A and the first cluster center B. If the prior cluster center 3 matches the first cluster center C, the prior cluster center 4 can only match the first cluster center D. Through the above matching method, n matching combinations can be obtained, n = 4*3*2*1. Under each matching combination, the sum of the distances between the four groups of prior cluster centers and the first cluster center is calculated, and the optimal matching combination with the smallest sum of distances is determined. According to the corresponding relationship in the optimal matching combination, the second cluster area matching each prior cluster area can be determined.

[0136] Continue reading Figure 3A , and continue with the above step 103 for explanation.

[0137] In step 104, at least one first cluster region is merged with the second cluster region to obtain a third cluster region matching the priori cluster region.

[0138] In some cases, if when performing the first clustering processing on the point set data, the M first clustering regions obtained are greater than the number of prior clustering regions, there may be a situation where the prior clustering region or the expanded prior clustering region includes multiple first clustering centers. At this time, in order to accurately determine the data points that match the prior clustering region, a merging operation is required.

[0139] In some embodiments, see Figure 3C , step 104 can be implemented by following steps 1041 to 1043.

[0140] In step 1041, based on a first position deviation threshold corresponding to the target environment, the range of the priori clustering region is expanded to obtain an expanded first priori clustering region.

[0141] In an embodiment of the present application, due to reasons such as hardware equipment or software algorithm, there is a certain data offset between the initial point set data collected for the target environment and the objects in the target environment. Therefore, a first position deviation threshold can be determined based on the empirical deviation amount obtained based on historical experience data, so as to adjust the prior clustering area based on the first position offset threshold, and then redetermine a third clustering area that matches the prior clustering area based on the adjusted prior clustering area.

[0142] As an example, if the a priori clustering area is represented by a polygon, the side length of the a priori clustering area can be increased by the first position deviation threshold; if the a priori clustering area is represented by a circle, the radius or diameter of the a priori clustering area can be increased by the first position deviation threshold. The specific expansion method can be determined based on the display form of the a priori clustering area, and is not limited again.

[0143] As an example, see Figure 6B , Figure 6B Displayed corresponding Fig. 6A The first priori clustering region is expanded from the four priori clustering regions in , and the dotted line indicates the boundary of the expanded first priori clustering region.

[0144] In step 1042, cluster center detection is performed on the expanded first priori clustering area to obtain a first detection result.

[0145] After expanding each priori clustering area based on the first position deviation threshold, cluster center detection is performed on the expanded first priori clustering area, that is, whether the first cluster center corresponding to each first clustering area falls within the range of the expanded first priori clustering area is detected to obtain a first detection result.

[0146] In step 1043, when the first detection result indicates that the expanded first priori clustering area includes at least one other cluster center and the second cluster center, the first clustering area corresponding to the at least one other cluster center is merged with the second clustering area to obtain a third clustering area matching the priori clustering area.

[0147] The other cluster centers are cluster centers other than the second cluster center among the first cluster centers corresponding to the M first cluster regions, and the second cluster center is the center of the second cluster region.

[0148] For example, see Fig. 7A , Fig. 7A Three a priori clustering regions pre-calibrated for a specified object are shown, and M first clustering regions obtained after the first clustering process is performed on the point set data, where M=7. Fig. 7AIn the example, the first cluster region b is the second cluster region corresponding to the priori cluster region 601, the first cluster region d is the second cluster region corresponding to the priori cluster region 602, and the first cluster region g is the second cluster region corresponding to the priori cluster region 603. If the first position deviation threshold is determined to be 2 cm based on the empirical deviation amount, the side length of each priori cluster region can be increased by 2 cm to expand each priori cluster region. Figure 7B , Figure 7B The figure shows the range of each priori clustering area expanded based on the first position deviation threshold. Figure 7B It can be seen that after the cluster center detection is performed on the expanded first a priori clustering region, the first detection result indicates that the first a priori clustering region after the expansion of the a priori clustering region 601 includes the second cluster center b and the first cluster center c (i.e., other cluster centers), the second cluster center b is the center of the second clustering region (corresponding to the first clustering region b) corresponding to the a priori clustering region 601, and the first cluster center c is the center of the first clustering region c. In this case, the first clustering region c corresponding to the first clustering center c and the second clustering region corresponding to the second clustering center b can be merged to obtain the third clustering region matched by the a priori clustering region 601.

[0149] Continue reading Figure 3A , and continue with the above step 104 for explanation.

[0150] In step 105, the third cluster center of the third cluster area is used as the clustering initial point, and the point set data is subjected to a second clustering process to obtain a first target cluster area that matches the priori cluster area.

[0151] After determining the third clustering area that matches each priori clustering area, determine the center of the third clustering area as the third clustering center. Use the third clustering center as the clustering initial point, cluster the point set data again, i.e., the second clustering process, and obtain the first target clustering area centered on the third clustering center.

[0152] As an example, see Figure 7BAfter obtaining the third clustering region for the prior clustering region 601, since the third clustering region includes two clustering regions (the second clustering region b and the first clustering region c), and the center of the third clustering region is located outside the two clustering regions, it can be seen that some data points located between the two clustering regions may also correspond to some data points of the prior clustering region 601. Therefore, the third clustering center of the third clustering region can be used as the clustering initial point to perform the second clustering processing, and the clustering region centered on the third clustering center is obtained as the first target clustering region matching the prior clustering region 601. The data points included in the first target clustering region are the data points that match the prior clustering region 601, which can be understood as the data points in the first target clustering region are generated for the specified object corresponding to the prior clustering region 601.

[0153] Applying the above embodiment, the point set data of the target environment is subjected to a first clustering process to obtain M first clustering regions, and the corresponding second clustering regions are pre-matched from the M first clustering regions for the a priori clustering regions pre-calibrated for the target environment, and the second clustering regions and at least one first clustering region are merged to obtain a third clustering region that matches each a priori clustering region, and then, the point set data is subjected to a second clustering process using the third clustering center of the third clustering region as the clustering initial point, thereby obtaining a first target clustering region that matches the a priori clustering region. Since the embodiment of the present application performs two clustering processes on the point set data, and the second clustering is performed based on the pre-matching and merging of the clustering regions after the first clustering, the second clustering can obtain a more accurate clustering result, so that the data points in the same clustering region are more highly correlated, which can improve the clustering accuracy of the first target clustering region, thereby improving the matching accuracy between the first target clustering region and the a priori clustering region.

[0154] In some embodiments, when the first detection result indicates that the expanded first prior clustering area does not include other cluster centers, the second cluster center of the second clustering area can also be used as the clustering initial point to perform a second clustering process on the point set data to obtain a second target clustering area that matches the prior clustering area.

[0155] For actual implementation, see Figure 7B, for the priori clustering region 603, the first detection result of the expanded first priori clustering region indicates that the expanded first priori clustering region only includes the second cluster center g, and does not include other cluster centers other than the second cluster center g. In this case, the second cluster center g can be directly used as the clustering initial point to perform the second clustering process to obtain the second target clustering region matching the priori clustering region, and the data points included in the second target clustering region are the data points matching the priori clustering region 603. In this case, the second clustering region corresponding to the priori clustering region can be regarded as the third clustering region of the priori clustering region.

[0156] In some embodiments, when the first detection result indicates that the second cluster center of the second cluster region is outside the range of the expanded first a priori cluster region, the matching relationship between the second cluster region and the a priori cluster region may also be cancelled.

[0157] For actual implementation, see Figure 7B For the prior clustering area 602, the first detection result of the expanded first prior clustering area indicates that the second cluster center d of the second clustering area d matching the prior clustering area 602 is located outside the range of the expanded first prior clustering area, which means that the offset between the second clustering area d and the prior clustering area 602 has exceeded the empirical offset, and the data points in the second clustering area d are not generated for the specified object corresponding to the prior clustering area 602. In this case, the matching relationship between the second clustering area d and the prior clustering area 602 can be directly cancelled.

[0158] In some embodiments, after each a priori clustering region is expanded based on the first position deviation threshold, the cluster center detection is performed on the expanded first a priori clustering region to obtain a first detection result. Based on the first detection result, it is determined whether the second clustering region corresponding to each a priori clustering region needs to cancel the matching relationship, merge, retain, etc., and according to the processing process when the first detection result indicates different results in the above embodiment, the third clustering region corresponding to each a priori clustering region is obtained, and the third cluster center of the third clustering region is used as the clustering initial point of the second clustering processing, and the point set data is subjected to a second clustering processing to obtain the target clustering region (the first target clustering region and the second target clustering region) corresponding to each a priori clustering region.

[0159] In some embodiments, after performing the second clustering process on the point set data, a secondary merging process of the clustering regions may also be performed in the following manner, which is described in detail below.

[0160] Obtain the intermediate clustering area obtained by performing the second clustering processing on the point set data; based on the second position deviation threshold corresponding to the target environment, expand the range of the priori clustering area to obtain the expanded second priori clustering area; perform cluster center detection on the expanded second priori clustering area to obtain a second detection result; when the second detection result indicates that the expanded second priori clustering area includes the intermediate cluster centers of multiple intermediate clustering areas, merge the multiple intermediate clustering areas to obtain a merged clustering area; based on the merged clustering area, determine the first target clustering area that matches the priori clustering area. The second position deviation threshold is greater than the first position deviation threshold, and the first position deviation threshold is the position deviation threshold used when merging at least one first clustering area with the second clustering area.

[0161] In actual implementation, during the second clustering process of the point set data, due to the clustering of data points in the point set, other cluster centers besides the third cluster center may appear. Therefore, the intermediate cluster area obtained by the second clustering process of the point set data can be obtained, and the intermediate cluster area can be merged.

[0162] Specifically, the second position deviation threshold is determined, and the second position deviation threshold is greater than the first position deviation threshold. This is because the second clustering area has been merged once, and the range of the intermediate clustering area obtained during the second clustering process is relatively larger than the range of the second clustering area, or the position of the intermediate clustering center of the intermediate clustering area is relatively farther away from the priori clustering center. The priori clustering area is expanded by the second position deviation threshold to obtain the expanded second priori clustering area corresponding to the priori clustering area. Similarly, the clustering center detection is performed on the expanded second priori clustering area to obtain the second detection result. If the second detection result indicates that the expanded second priori clustering area includes the intermediate clustering centers of multiple intermediate clustering areas, the multiple intermediate clustering areas are merged to obtain the merged clustering area. Here, the merged clustering area can be directly used as the first target clustering area matched by the priori clustering area, or the fourth clustering center of the merged clustering area can be used as the clustering initial point, and the point set data is iteratively clustered until the iterative clustering end condition is met to obtain the first target clustering area of ​​the priori clustering area.

[0163] In some embodiments, the above “determining a first target clustering region matching the priori clustering region based on the merged clustering region” can be implemented in the following manner.

[0164] The fourth cluster center of the merged cluster area is used as the clustering initial point, and the point set data is iteratively clustered to obtain the first target cluster area that matches the priori cluster area.

[0165] Among them, the iterative clustering termination conditions of the iterative clustering processing include at least one of the following: the number of clustering times reaches a preset number, the repetition rate of the data points included in the clustering area obtained by this clustering and the data points included in the clustering area obtained by the previous clustering reaches a preset repetition rate, and the position deviation between the cluster center of the clustering area obtained by this clustering and the cluster center of the clustering area obtained by the previous clustering is less than a preset deviation.

[0166] The fourth cluster center is the center of the merged cluster area. In the process of iterative clustering of point set data, the cluster center of the cluster area obtained by this clustering can be used as the clustering initial point for the next clustering, until the iterative clustering end condition is met, and the cluster area obtained by the last clustering is used as the first target cluster area for matching the priori cluster area.

[0167] For example, see Figure 8 , Figure 8 For the corresponding Figure 5B The schematic diagram of the target clustering area obtained after the point set data is subjected to the first clustering process and the second clustering process, wherein the target clustering area includes the first target clustering area and the second target clustering area. Figure 8 , the prior clustering region 5011 matches the target clustering region 1, the prior clustering region 5021 matches the target clustering region 2, the prior clustering region 5031 matches the target clustering region 3, and the prior clustering region 5041 matches the target clustering region 4.

[0168] In a specific embodiment, in order to solve the impact of random deviation of eye movement data collected by an eye tracker in any direction within a small range near the expected position, the embodiment of the present application can cluster and region match the eye movement data through the above-mentioned region matching method to reduce the negative impact of data deviation and improve the matching accuracy of eye movement data and the expected position.

[0169] like Fig. 9 As shown in the figure, the eye tracker is below the screen, tilted upward to shoot the image of the human eye, and the human eye looks at the screen, and the eye tracker can obtain the position of the human eye on the screen. However, due to the accuracy of the eye tracker itself, the eye viewing position (eye movement data) obtained by the eye tracker may be offset from the actual viewing position.

[0170] by Figure 4A For example, suppose Fig. 9 The screen shown shows Figure 4A The image shown, Figure 4AThe positions and sizes of the four objects (specified objects) in the image are known. When the human eye watches the screen for a period of time, the eye tracker can obtain point set data (eye movement data) of certain positions on the screen. For example, if the frame rate of the eye tracker is 120Hz, the eye tracker will obtain 120 data points per second without considering the eye moving out of the screen. When the human eye moves for a period of time, a series of moving data points will be obtained, such as Figure 5A Initial eye-tracking data shown.

[0171] pass Figure 5A It can be easily seen that the human eye sees the Figure 4A Although the object in the lower left corner is viewed, the acquired eye movement data is overall offset upward, and the eye movement data in the lower right corner also has a certain amount of random jitter and offset, which is outside the range of the actual expected priori clustering area 5041. That is, there is a problem of data offset between the actual initial eye movement data collected and the expected data distribution.

[0172] If you want to confirm Figure 5A In order to determine which data points are generated by human eyes looking at objects in the corresponding prior clustering area, it is necessary to cluster and match the initial point set data with jitter and random offset.

[0173] First, the initial point set data can be de-jittered, that is, position smoothing can be performed.

[0174] As an example, the eye movement data captured by the eye tracker within a specified period of time can be obtained. For the eye movement data with time series information, the data points are arranged in time series order, and multiple multi-frame smoothing is used to remove the jitter of the data points. The idea is to use the R surrounding data points of each frame data point for position smoothing, use the first initial position information of the initial data point of the current frame before updating, the first updated position information of the n surrounding data points that have completed position smoothing among the R surrounding data points, and the second initial position information of the m surrounding data points that have not completed position smoothing among the R surrounding data points, and assign corresponding weights to each position information, perform position smoothing on the initial data point of the current frame by weighted summation, and calculate the second updated position information of the initial data point of the current frame after position smoothing.

[0175] The maximum weight can be given to the data of the initial data point of the current frame before the update, and the updated data among the R surrounding data points can be given a larger weight, and the closer to the initial data point of the current frame, the larger the weight, so as to give weight to each position information, so as to solve the problem of initial data value jitter and subsequent large-scale jitter. The weight coefficient can be adjusted, and the number of surrounding frames used can also be adjusted. Specifically, corresponding weights can be assigned to the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points in such a manner that the weight corresponding to the first initial position information of the initial data point of the current frame before position smoothing is the largest, among the R related initial data points, the weight corresponding to the first updated position information of the n surrounding data points after position smoothing is greater than the weight corresponding to the second initial position information of the m surrounding data points before position smoothing, and the weight corresponding to the first updated position information of the surrounding data points that are closer in timing to the initial data point of the current frame among the n surrounding data points is greater, and the first initial position information is assigned a first weight, the second initial position information of the m surrounding data points is assigned a second weight, and the first updated position information of the n surrounding data points is assigned a third weight, the first weight is greater than the second weight, and the third weight is greater than the second weight.

[0176] As an example, the second updated position information of the initial data point of the current frame after position smoothing is calculated by the following formula. In the formula, P represents an initial data point. The lower right subscript of P indicates the time sequence number of this initial data point, and the upper right subscript of P indicates whether to use the data before or after the update of this initial data point (i.e., the data before or after position smoothing). If it is marked new, it indicates the updated data, and if it is marked old, it indicates the data before the update.

[0177] When the sequence number of the initial data point of the current frame is greater than 3, for example, when the initial data point of the current frame is the initial data point of the 24th frame, the second updated position information of the initial data point of the 24th frame is calculated using the following formula:

[0178]

[0179] in, represents the second updated position information of the initial data point with the time sequence number of 24, represents the first updated position information of the initial data point with the time sequence number 21 (the corresponding third weight is 0.1), represents the first updated position information of the initial data point with the time sequence number of 22 (the corresponding third weight is 0.2), represents the first updated position information of the initial data point with the time sequence number of 23 (the corresponding third weight is 0.3), Represents the first initial position information of the initial data point with the time series number of 24 (the corresponding first weight is 0.4).

[0180] The second updated position information of the initial data point of the first frame is calculated using the following formula:

[0181]

[0182] in, Indicates the second updated position information of the initial data point with the time sequence number 1, represents the first initial position information of the initial data point with the time series number 1 (the corresponding first weight is 0.4), represents the second initial position information of the initial data point with the time series number of 2 (the corresponding second weight is 0.3), represents the second initial position information of the initial data point with the time series number of 3 (the corresponding second weight is 0.2), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0183] The second updated position information of the initial data point of the second frame is calculated using the following formula:

[0184]

[0185] in, represents the second updated position information of the initial data point with the time sequence number 2, represents the first updated position information of the initial data point with the time sequence number 1 (the corresponding third weight is 0.3), represents the first initial position information of the initial data point with the time series number of 2 (the corresponding first weight is 0.4), represents the second initial position information of the initial data point with the time series number of 3 (the corresponding second weight is 0.2), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0186] The second updated position information of the initial data point of the third frame is calculated using the following formula:

[0187]

[0188] in, represents the second updated position information of the initial data point with the time sequence number 3, represents the first updated position information of the initial data point with the time sequence number 1 (the corresponding third weight is 0.2), represents the first updated position information of the initial data point with the time sequence number of 2 (the corresponding third weight is 0.3), represents the first initial position information of the initial data point with the time series number of 3 (the corresponding first weight is 0.4), Represents the second initial position information of the initial data point with time series number 4 (the corresponding second weight is 0.1).

[0189] The initial data points of other frames are calculated according to their time series numbers and the corresponding formulas mentioned above to obtain the second updated position information after the position update. For each specific smoothing, the following method can be used: for the nth frame data, the data before the nth update, the data after the n-1th update, the data after the n-2th update, the data after the n-3th update, and other frames of data can be combined according to different weights. That is, the past data is used for weight combination. However, for the initial data update, the future data is used for different weight combinations. This reduces the situation where the de-jitter effect is poor when the deviation between the initial data point and the data point after position smoothing is large.

[0190] See also Figure 5B , Figure 5B For Figure 5A The eye movement data after position smoothing of the initial eye movement data in , and the eye movement data after jitter removal, still have position offset relative to the prior clustering area of ​​the real viewed object. Therefore, all eye movement data can be clustered to obtain the clustering points of the eye movement data distribution.

[0191] Use the Figure 4A The four specified objects shown in the figure are pre-calibrated in four a priori clustering areas. Ideally, most of the eye movement data points will be clustered in the four areas corresponding to the a priori clustering areas, that is, the number of ideal clustering areas is known. Clustering algorithms such as K-means algorithm, K-means++ algorithm, Gaussian mixture model, etc. can be used to cluster all eye movement data points. According to the actual clustering situation, you can choose to cluster in the way that the center point of the a priori clustering area is the clustering initial point, or you can cluster all eye movement data points for a given number of clustering areas to obtain 4 first clustering center points (corresponding to 4 first clustering areas).

[0192] like Fig. 6AAs shown, the priori cluster center corresponding to each priori cluster area and the first cluster center corresponding to each first cluster area are determined, the distance from the current priori cluster center to any first cluster center and the distance from the next current priori cluster center to the other remaining first cluster centers are calculated, a first cluster center is matched for each priori cluster center, and the distance between the matched first cluster center and the priori cluster center is calculated. For example, for priori cluster center 1, there are four first cluster centers that can be matched for it. If the priori cluster center matches the first cluster center A, the first cluster centers that can be matched by priori cluster center 2 are the other three first cluster centers except the first cluster center A. If the priori cluster center 2 matches the first cluster center B, the first cluster centers that can be matched by priori cluster center 3 are the other two first cluster centers except the first cluster center A and the first cluster center B. If the first cluster center C is matched for the priori cluster center 3, the priori cluster center 4 can only match the first cluster center D. Through the above matching method, n matching combinations can be obtained, n = 4*3*2*1. Under each matching combination, the sum of the distances between the four groups of prior cluster centers and the first cluster center is calculated, and the optimal matching combination with the smallest sum of distances is determined. According to the corresponding relationship in the optimal matching combination, the prior cluster area is matched with the corresponding first cluster area, and the second cluster area matched by each prior cluster area is obtained.

[0193] Then, according to the range of the prior clustering area and the offset of the eye movement data, the first clustering area matching each prior clustering area is screened, wherein the offset is an empirical offset determined by the historical data of the eye tracker. Based on the offset of the eye tracker, a suitable first position offset threshold is determined, and each prior clustering area is expanded according to the first position offset threshold to obtain an expanded first prior clustering area. Cluster point detection is performed on each expanded first prior clustering area to obtain a first detection result.

[0194] Determine whether the second cluster center of the second cluster area that matches the priori cluster area is located in the expanded first priori cluster area according to the first detection result. Delete the second cluster center that is outside the scope of the corresponding expanded first priori cluster area, that is, cancel the matching relationship between the second cluster area corresponding to the second cluster center and the corresponding priori cluster area. Here, the second cluster center is outside the scope of the corresponding expanded first priori cluster area, indicating that the distance between the second cluster center and the corresponding priori cluster center is greater than a certain distance threshold, that is, the actual position offset of the eye movement data point in the second cluster area corresponding to the second cluster center corresponding to the specified object of the priori cluster area is too large, and the eye movement data in the second cluster area is not generated for the specified object, so the matching relationship is canceled. In this case, after canceling the matching relationship between the second cluster area and the priori cluster area, all eye movement data points in the second cluster area can also be deleted and no longer participate in subsequent clustering. Assume that the number of second cluster areas whose matching relationship is canceled in accordance with this detection result is x.

[0195] According to the first detection result, the expanded first a priori clustering region includes the second cluster center of the corresponding second clustering region and other cluster centers. The other cluster centers are the cluster centers of the first cluster centers corresponding to the four first clustering regions, except the second cluster center. The second clustering region corresponding to the second cluster center is merged with the first clustering regions corresponding to the other cluster centers to obtain the third clustering region corresponding to the a priori clustering region, and the third cluster center of the third clustering region is calculated, and the position of the third cluster center is saved. Assume that the number of second clustering regions that meet this detection result and are merged is y.

[0196] After the above merging and deleting process, the cluster centers corresponding to each priori clustering region are determined. There are two cases of the cluster center, including the third cluster center determined after the merging process and the second cluster center without any processing. The cluster centers corresponding to each priori clustering region are used as the clustering initial points of the secondary clustering. The eye movement data is subjected to the second clustering process using the clustering method with the clustering initial points to obtain 4-xy intermediate clustering regions. The cluster centers of the intermediate clustering regions will move in a small range near the original position, and the cluster centers with the merged data will be more accurate.

[0197] Determine the second position deviation threshold, merge or delete the intermediate clustering areas. The process is similar to the above steps, but the second position deviation threshold is slightly larger than the first position deviation threshold. After merging or deleting the intermediate clustering areas, the target clustering areas corresponding to each priori clustering area can be directly obtained, or the iterative clustering process can be continued until the iterative clustering end condition is met to obtain the target clustering areas corresponding to each priori clustering area. Figure 8, Figure 8 The visualization results after two clustering and region adjustment of the eye movement data are shown. The rectangular box in the figure is the prior clustering area, which is the expected area of ​​the specified object; the circular area is the target clustering area, which is the actual area of ​​the eye movement data. According to the matching results determined above, accurate matching between unequal areas is achieved in the case of eye movement data offset.

[0198] The following continues to describe the exemplary structure of the region matching device 455 in the environment provided by the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the area matching device 455 in the environment of the memory 440 may include:

[0199] The acquisition module 4551 is used to acquire point set data of the target environment, wherein the point set data includes N data points.

[0200] The first clustering module 4552 is used to perform the first clustering processing on N data points to obtain M first clustering regions, wherein each first clustering region corresponds to a first clustering center, each first clustering region contains P data points, N, M, and P are positive integers, and P≤N.

[0201] The pre-matching module 4553 is used to determine a second clustering region from the M first clustering regions that matches a pre-calibrated priori clustering region.

[0202] The merging module 4554 is used to merge at least one first clustering region with the second clustering region to obtain a third clustering region that matches the priori clustering region.

[0203] The second clustering module 4555 is used to perform a second clustering process on the point set data using the third cluster center of the third cluster area as the clustering initial point to obtain a first target cluster area that matches the priori cluster area.

[0204] In some embodiments, the first clustering module 4552 is also used to use the prior cluster center of the prior clustering area as the clustering initial point, perform a first clustering process on the N data points in the point set data, and obtain M first clustering areas; or use the number of prior clustering areas as the number of clustering areas, perform a first clustering process on the N data points in the point set data, and obtain M first clustering areas, where M is equal to the number of prior clustering areas.

[0205] In some embodiments, the pre-matching module 4553 is also used to determine the distance between the prior cluster center of the prior cluster area and each first cluster center; from the M first cluster areas, the first cluster area with the smallest distance is screened out, and the screened out first cluster area with the smallest distance is used as the second cluster area matching the prior cluster area.

[0206] In some embodiments, the merging module 4554 is also used to expand the range of the prior clustering area based on the first position deviation threshold corresponding to the target environment to obtain an expanded first prior clustering area; perform cluster center detection on the expanded first prior clustering area to obtain a first detection result; when the first detection result indicates that the expanded first prior clustering area includes at least one other cluster center and a second cluster center, merge the first clustering area corresponding to at least one other cluster center with the second clustering area to obtain a third clustering area matching the prior clustering area, wherein the other cluster centers are cluster centers other than the second cluster center among the first cluster centers corresponding to the M first clustering areas, and the second cluster center is the center of the second clustering area.

[0207] In some embodiments, the area matching device 455 in the environment also includes a third clustering module, which is used to perform a second clustering processing on the point set data using the second cluster center of the second cluster area as the clustering initial point when the first detection result indicates that the expanded first prior clustering area does not include other cluster centers, so as to obtain a second target clustering area that matches the prior clustering area.

[0208] In some embodiments, the region matching device 455 in the environment further includes a matching cancellation module for canceling the matching relationship between the second cluster region and the priori cluster region when the first detection result indicates that the second cluster center of the second cluster region is outside the range of the expanded first priori cluster region.

[0209] In some embodiments, the second clustering module 4555 is also used to obtain an intermediate clustering area obtained by performing the second clustering processing on the point set data after performing the second clustering processing on the point set data; based on the second position deviation threshold corresponding to the target environment, the range of the prior clustering area is expanded to obtain an expanded second prior clustering area; cluster center detection is performed on the expanded second prior clustering area to obtain a second detection result; when the second detection result indicates that the expanded second prior clustering area includes intermediate clustering centers of multiple intermediate clustering areas, the multiple intermediate clustering areas are merged to obtain a merged clustering area; wherein the second position deviation threshold is greater than the first position deviation threshold, and the first position deviation threshold is the position deviation threshold used when merging at least one first clustering area with the second clustering area; based on the merged clustering area, a first target clustering area matching the priori clustering area is determined.

[0210] In some embodiments, the second clustering module 4555 is also used to use the fourth cluster center of the merged cluster area as the clustering initial point, perform iterative clustering processing on the point set data, and obtain a first target clustering area that matches the prior clustering area; wherein, the iterative clustering termination conditions of the iterative clustering processing include at least one of the following: the number of clustering times reaches a preset number, the repetition rate of the data points included in the cluster area obtained by this clustering and the data points included in the cluster area obtained by the previous clustering reaches a preset repetition rate, and the position deviation between the cluster center of the cluster area obtained by this clustering and the cluster center of the cluster area obtained by the previous clustering is less than a preset deviation.

[0211] In some embodiments, the acquisition module 4551 is also used to acquire initial point set data generated for the target environment, wherein the initial point set data includes N initial data points; and position smoothing processing is performed on the N initial data points in the initial point set data to obtain the point set data.

[0212] In some embodiments, the acquisition module 4551 is also used to sort the N initial data points based on the timing information of each initial data point; for each initial data point, perform the following processing: based on the sorting result, determine the R surrounding data points associated with the initial data point, and the timing difference between the surrounding data points and the initial data point is less than a preset threshold; determine n surrounding data points that have completed position smoothing and m surrounding data points that have not completed position smoothing among the R surrounding data points, where R, m, and n are all integers, and m=Rn; obtain the first initial position information of the initial data point, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points; based on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points, perform position smoothing processing on the initial data point to obtain the second updated position information of the initial data point; use the second updated position information of each initial data point in the initial point set data as the position information of each data point in the point set data.

[0213] In some embodiments, the acquisition module 4551 is also used to determine, based on the sorting result, R surrounding data points adjacent to and before the initial data point when the timing sequence number of the initial data point is greater than the first specified sequence number; when the timing sequence number of the initial data point is greater than the second specified sequence number and less than or equal to the first specified sequence number, based on the sorting result, n surrounding data points adjacent to and before the initial data point, and m surrounding data points adjacent to and after the initial data point; when the timing sequence number of the initial data point is equal to the second specified sequence number, based on the sorting result, R surrounding data points adjacent to and after the initial data point are determined.

[0214] In some embodiments, the acquisition module 4551 is also used to obtain a first weight corresponding to the first initial position information, a second weight corresponding to the second initial position information of the m surrounding data points, and a third weight corresponding to the first updated position information of the n surrounding data points, wherein the first weight is greater than the second weight, and the third weight is greater than the second weight; based on the first weight, the second weight and the third weight, the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points are weightedly summed to obtain the second updated position information of the initial data points.

[0215] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device 400 reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device 400 performs the area matching method in the environment described in the embodiment of the present application.

[0216] The embodiment of the present application provides a computer-readable storage medium in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will be caused to execute the region matching method in the environment provided by the embodiment of the present application, for example, Figure 3A Region matching method in the environment shown.

[0217] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0218] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0219] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0220] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0221] Applying the above embodiment, the point set data of the target environment is subjected to a first clustering process to obtain M first clustering regions, and the corresponding second clustering regions are pre-matched from the M first clustering regions for the a priori clustering regions pre-calibrated for the target environment, and the second clustering regions and at least one first clustering region are merged to obtain a third clustering region that matches each a priori clustering region, and then, the point set data is subjected to a second clustering process using the third clustering center of the third clustering region as the clustering initial point, thereby obtaining a first target clustering region that matches the a priori clustering region. Since the embodiment of the present application performs two clustering processes on the point set data, and the second clustering is performed based on the pre-matching and merging of the clustering regions after the first clustering, the second clustering can obtain a more accurate clustering result, so that the data points in the same clustering region are more highly correlated, which can improve the clustering accuracy of the first target clustering region, thereby improving the matching accuracy between the first target clustering region and the a priori clustering region.

[0222] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for matching regions in an environment, characterized in that: The method comprises: Acquire point set data of a target environment, wherein the point set data includes N data points; Performing a first clustering process on the N data points to obtain M first clustering regions, wherein each of the first clustering regions corresponds to a first clustering center, each of the first clustering regions contains P data points, N, M, and P are positive integers, and P≤N; Determine a second clustering region matching a pre-calibrated priori clustering region from the M first clustering regions; Merging at least one of the first clustering regions with the second clustering region to obtain a third clustering region matching the priori clustering region; The third cluster center of the third cluster area is used as the clustering initial point, and the point set data is subjected to a second clustering process to obtain a first target cluster area that matches the priori cluster area.

2. The method according to claim 1, characterized in that: The performing a first clustering process on the N data points to obtain M first clustering regions includes: Taking the a priori cluster center of the a priori cluster region as the clustering initial point, performing a first clustering process on the N data points in the point set data to obtain M first cluster regions; or Taking the number of the priori clustering regions as the number of clustering regions, a first clustering process is performed on the N data points in the point set data to obtain M first clustering regions, where M is equal to the number of the priori clustering regions.

3. The method according to claim 1, characterized in that The determining, from the M first clustering regions, a second clustering region matching the priori clustering region comprises: Determining the distance between the priori cluster center of the priori cluster area and each of the first cluster centers; The first cluster region with the smallest distance is screened out from the M first cluster regions, and the screened out first cluster region with the smallest distance is used as the second cluster region matching the priori cluster region.

4. The method according to claim 1, characterized in that: The step of merging at least one of the first clustering regions with the second clustering region to obtain a third clustering region matching the priori clustering region includes: Based on a first position deviation threshold corresponding to the target environment, the range of the priori clustering area is expanded to obtain an expanded first priori clustering area; Performing cluster center detection on the expanded first a priori clustering region to obtain a first detection result; When the first detection result indicates that the expanded first prior clustering area includes at least one other cluster center and a second cluster center, the first clustering area corresponding to the at least one other cluster center is merged with the second clustering area to obtain a third clustering area matching the prior clustering area, wherein the other cluster centers are cluster centers of the first cluster centers corresponding to the M first clustering areas except the second cluster center, and the second cluster center is the center of the second clustering area.

5. The method according to claim 4, characterized in that When the first detection result indicates that the expanded first a priori clustering area does not include the other clustering centers, the method further includes: Taking the second cluster center of the second cluster area as the clustering initial point, a second clustering process is performed on the point set data to obtain a second target cluster area that matches the priori cluster area.

6. The method according to claim 4, characterized in that When the first detection result indicates that the second cluster center of the second cluster area is located outside the range of the expanded first a priori cluster area, the method further includes: The matching relationship between the second clustering area and the priori clustering area is cancelled.

7. The method according to claim 1, characterized in that After performing the second clustering process on the point set data, the method further includes: Acquire an intermediate clustering region obtained by performing the second clustering process on the point set data; Based on a second position deviation threshold corresponding to the target environment, the range of the priori clustering area is expanded to obtain an expanded second priori clustering area; Performing cluster center detection on the expanded second priori clustering region to obtain a second detection result; When the second detection result indicates that the expanded second a priori clustering region includes intermediate clustering centers of a plurality of the intermediate clustering regions, merging the plurality of the intermediate clustering regions to obtain a merged clustering region; The second position deviation threshold is greater than the first position deviation threshold, and the first position deviation threshold is a position deviation threshold used when merging at least one of the first clustering regions with the second clustering region; Based on the merged cluster regions, a first target cluster region matching the priori cluster region is determined.

8. The method according to claim 7, characterized in that The determining, based on the merged cluster regions, a first target cluster region matching the priori cluster region comprises: Taking the fourth cluster center of the merged cluster area as the cluster initial point, performing iterative clustering processing on the point set data to obtain a first target cluster area that matches the priori cluster area; Among them, the iterative clustering termination conditions of the iterative clustering processing include at least one of the following: the number of clustering times reaches a preset number, the repetition rate of the data points included in the clustering area obtained by this clustering and the data points included in the clustering area obtained by the previous clustering reaches a preset repetition rate, and the position deviation between the cluster center of the clustering area obtained by this clustering and the cluster center of the clustering area obtained by the previous clustering is less than a preset deviation.

9. The method according to claim 1, characterized in that: The step of obtaining point set data of the target environment includes: Acquire initial point set data generated for a target environment, wherein the initial point set data includes N initial data points; Position smoothing processing is performed on the N initial data points in the initial point set data to obtain the point set data.

10. The method according to claim 9, characterized in that The performing position smoothing processing on the N initial data points in the initial point set data to obtain the point set data includes: Sorting the N initial data points based on the time series information of each of the initial data points; For each of the initial data points, the following processing is performed: Based on the sorting result of the sorting, determine R surrounding data points associated with the initial data point, and the time sequence difference between the surrounding data points and the initial data point is less than a preset threshold; Determine n surrounding data points that have completed position smoothing and m surrounding data points that have not completed position smoothing among the R surrounding data points, wherein R, m, and n are all integers, and m=Rn; Acquire first initial position information of the initial data point, second initial position information of the m surrounding data points, and first updated position information of the n surrounding data points; Based on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points, performing position smoothing processing on the initial data point to obtain the second updated position information of the initial data point; The second updated position information of each of the initial data points in the initial point set data is used as the position information of each data point in the point set data.

11. The method according to claim 10, characterized in that The step of determining R surrounding data points associated with the initial data point based on the sorting result includes: When the time sequence number of the initial data point is greater than the first specified sequence number, based on the sorting result, R surrounding data points adjacent to the initial data point and located before the initial data point are determined; When the time sequence number of the initial data point is greater than the second specified sequence number and less than or equal to the first specified sequence number, based on the sorting result, n surrounding data points adjacent to the initial data point and located before the initial data point, and m surrounding data points adjacent to the initial data point and located after the initial data point are determined; When the time sequence number of the initial data point is equal to the second designated sequence number, based on the sorting result, R surrounding data points adjacent to the initial data point and located after the initial data point are determined.

12. The method according to claim 10, characterized in that The step of performing position smoothing processing on the initial data point based on the first initial position information, the second initial position information of the m surrounding data points, and the first updated position information of the n surrounding data points to obtain the second updated position information of the initial data point includes: Obtaining a first weight corresponding to the first initial position information, a second weight corresponding to the second initial position information of the m surrounding data points, and a third weight corresponding to the first updated position information of the n surrounding data points, wherein the first weight is greater than the second weight, and the third weight is greater than the second weight; Based on the first weight, the second weight and the third weight, the first initial position information, the second initial position information of the m surrounding data points and the first updated position information of the n surrounding data points are weightedly summed to obtain the second updated position information of the initial data point.

13. A region matching device in an environment, characterized in that: The device comprises: An acquisition module, used to acquire point set data of a target environment, wherein the point set data includes N data points; A first clustering module, used for performing a first clustering process on the N data points to obtain M first clustering regions, wherein each of the first clustering regions corresponds to a first clustering center, each of the first clustering regions contains P data points, N, M, and P are positive integers, and P≤N; A pre-matching module, used to determine a second clustering region matching a pre-calibrated priori clustering region from the M first clustering regions; a merging module, configured to merge at least one of the first clustering regions with the second clustering region to obtain a third clustering region matching the priori clustering region; The second clustering module is used to perform a second clustering process on the point set data using the third cluster center of the third cluster area as a clustering initial point to obtain a first target cluster area that matches the priori cluster area.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, configured to implement the method according to any one of claims 1 to 12 when executing the computer executable instructions stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Citation Information

Cited By

  • Region matching method and apparatus in environment, and electronic device, computer-readable storage medium and computer program product

    WO2026020742A1